Case Study: Comix

With a Billboard Hot 100 artist to play at Coachella, Comix were looking to experiment with facial tracking software and real-time VFX (visual effects) rendering for their set.

Project Type Data Type Tools
Integration JSON
  • Python3
  • Sockets
  • PyInstaller

Comix had already two pieces of software in mind, which for this case study I’ll simply call FT (facial tracking) and RVFX (real-time visual effects). The issue, however, was connecting the two in a method that was easy to implement for their VFX artists and directors, given FT could not directly connect to RVFX.

The solution was a standalone executable that parsed JSON data being sent from FT’s TCP server and reformatted it to be sent across UDP to RVFX’s OSC server.

Anybody running the executable was prompted for ports of the TCP and OSC server, which was an acceptable level of technicality. I wanted to play around with argument passing on the command line so if the client wanted to automate this process out it was possible also as a relatively low-level solution.

Reflections
  • Software providers are your friends - your client is paying for their tools after all. Without documentation on a certain element of the FT’s message structure, I thought I could fumble my way to a solution taking me far longer than if I’d have been in contact with their developers since day 1.
  • Ensure your kit works before starting a project. Much like your software mocks you on demo day and decides to not work, your camera may decide to give up the ghost all together before you’ve even got cracking with a project that requires your camera.