Spark AR: FaceMask Masterclass
Make your customised facemasks using SPARK AR in under 10 minutes
Face Tracking and Augmented Reality, together have opened up a whole wide avenue for developers to try out their creativity, imagination and programming skills to create applications which have both an entertaining and commercial value.
One such use of Face Tracking in Augmented Reality is the creation of Face Filters, which has been made possible through the release of Software Development Tools like Spark AR, Lens Studio and Unity. Face Filters have been present as a mode of expression and entertaining people since past 4–5 years in applications like Facebook/Messenger, Snapchat and Instagram.
During the times of COVID -19 Epidemic, let us utilize this time for learning how to build Corona Prevention FaceMask using Spark AR and GIMP in less than 10 minutes. You can build your customized face masks using this tutorial as well.
Let’s Begin
Open Spark AR Studio and click on +New Project on the left bar and select Blank Project. It would take about half a minute for your project to initialize and load in the window.
Now click on the +Add Object button located at the bottom of the Scene area in left, and select Face Tracker and click Insert. Make sure that the Face Tracker object is the child of the Camera object.
We use Face Tracker to create an effect that responds to or augments someone’s face. When combined with Face Mesh it creates a surface that can detect facial movements and expressions.
Now add the Face Mesh to the Face Tracker, by right-clicking on Face Tracker and then clicking on add. So the setup for our FaceMask project is ready, now we move onto the Texture Part of our project.
Designing the FaceMask Texture
For designing, placing and resizing our FaceMask project, we can use any image graphics editing software like Photoshop, GIMP, Canva etc. In this tutorial, I would be using GIMP which is an open-source raster graphics editor used for image retouching and editing.
- Open the faceMasculine.jpg in the Textures folder in Face Reference Assets.
- Now open the image of the FaceMask you want to overlay on Face Tracking, in the GIMP Editor in another window.
- Select Layer from Menu Bar and click on Transparency, and click on Add Alpha Channel. Again Under Transparency, click on Color to Alpha to remove the white background from the image.
The alpha channel is a color component that represents the degree of transparency of a color (i.e., the red, green and blue channels). It is used to determine how a pixel is rendered when blended with another.
- Copy the image onto the faceMasculine.jpg in the previous window and resize and position it on the face mesh texture as you would like it to appear on the tracked face.
- After doing that head over to the section on the bottom right where you would be able to see the layers on the image. Click on the ‘eye’ icon before the faceMasculine.jpg image layer to make it invisible.
- Save the image to a suitable path.
Your FaceMask Texture is ready to be applied to the Face Mesh you created in the first step.
Wrapping it Up
We would close the GIMP Editor, and move to the Spark AR application for this step. Click on the Face Mesh in the Scene panel on the left and head over to the Attributes panel on the right. Under it, select Add material to add a material to the Face Mesh you have created.
We use textures and materials to define how objects look in our effects. Textures define detail and colour. Materials control how a texture is used, for example: How reflective or opaque a texture is.
It is time to add the FaceMask Texture we created in the previous step, for that go to Add Asset option in the bottom left of the Spark Editor. Select the image from the path where you saved it, you would create the required Texture in the Spark AR project this way, which can be seen under the Textures Folder in the Assets panel on the left.
Select material0 from the Assets panel and in the attributes section, click on Add Material and select the newly created FaceMask Texture. This would add the Texture to the Face Mesh Material and you can see the FaceMask, getting overlayed on the top of the face like the way you set it up in the GIMP editor.
Adding Audio (Optional)
In the Assets panel, click on Add Asset, then type and select the Audio Playback Controller. Now go to the Attributes panel and under Audio, select the audio file you want to play in your effect.
Spark AR supports Mono M4A Audio format, with a sampling frequency of 44.1kHz. To convert your MP3 Audio file to the required format you can use the given Online Tool or offline Softwares like Audacity.
In Spark AR we are able to play audio through the help of these audio controllers, which can be regulated with the help of Scripting or the Patch Editor.
In our effect, we are playing audio throughout the time the face is being tracked and our FaceMask is being overlayed on a person’s face. Hence we don’t need either of scripting or the patch editor. Click on Add Object in the Scene panel on the left and add Speaker object. Make the Speaker object the child of the faceTracker0 object you have already created. In the Attributes section, in Audio property select audioPlaybackController0.
Cool! So you have added a background music to your effect as well. Its time to try the effect out, which is covered in the next section.
Testing the Effect
Do test out your effect by connecting your mobile device to the PC, and opening the Spark AR Player in your mobile device. Then click on Test on Device option located on the bottom of the left Toolbar as shown in the picture. Then you can select the Send to Connected Device option. Enjoy sharing the Selfies and GIFs made using the FaceMask Filter.
Yay! So we were able to make our custom FaceMask effect using Audio within 10 minutes. 😎 😉
Feel free to check out the Github Repository for any help you require with the project or you want to clone the source project. Keep following me for upcoming tutorials on Spark AR, Happy Learning!
