Truly — there's no point.

A wonderfully pointless VR experience created with Tilt Brush and Unity. Record a 5-second GIF with hand puppets in VR!

Hop into Pointless Puppets.
Play with puppets.
Hold up some signs.
Start recording.
Act out a short dramatic scene.
And — voila! — a 5-sec GIF will appear on your Desktop!

This was a collaboration project with Chris Gallello (March 2017).

Pointless Puppets was featured on RockPaperShotgun! Read all about it.  

Virtual Reality

Interaction Design

Environment Design

Character Design

Graphic Design

Tilt Brush

Unity3D

Research & Development

Download this app (Vive)

day 1: friday

OBJECTIVES + Accomplishments

• Get the VR workflow set up for our in-VR collaboration weekend:
  • MasterpieceVR to do our initial brainstorming together in the same VR space
  • BigScreen to see our own Desktops while in VR and to share each other's desktops with each other, as we sit next to each other in virtual space.......... while also next to each other in IRL space........ 
  • Purple (Chris' startup) for project planning. 
• Plan the project:
  • What is it that we are making?
  • We came up with creating a short interactive story that involved a puppet show, straddling the line of what is/isn't reality. We really wanted to involve the user with some semblance of agency, breaking fourth walls. (We were very optimistic! lol) 
  • We got a very rough storyboard together
• We were able to make a basic recording of the Vive controller movements. 
• We got the Unity's EditorVR working.
• We set up a 360 camera to live stream our weekend. hahaha

Challenges

• I ran into major Vive setup issues around 4PM of our first day. Suddenly, the tracking for my Vive wonked out and I couldn't work in VR anymore because it was so nauseating. I ended up having to troubleshoot hardware issues for four hours, trying various base station setups, swapping in/out of my own Vive hardware and Chris'. We tried everything we could think of, covered all the reflective surfaces, turned off the lights, etc. At some point Chris' Vive started getting wonky too. Eventually, we consistently singled the issue down to one really strange section of the room. After four hours, we decided it was a ghost. (Truly. Chris' place has a friendly spirit. Likely intrigued by VR!!!)
• At some point, my desktop stopped booting up, for whatever reason. It was not a good hardware day for me. 

day 2: Saturday

OBJECTIVES + Accomplishments

• Estella's computer booted. This was indeed an accomplishment. I double-checked all my internal wire connections, but otherwise made no changes. It just didn't want to boot up on Friday and wanted to boot up on Saturday. 
• Tilt Brush rough painting: stage and characters
• Learned how to export from Tilt Brush and bring into Unity
• Approach to painting the Tilt Brush files:
  • I painted the assets altogether in the same VR file/canvas so I could keep the scale and design relationships intact. So I had one large piece with all the pieces mostly in place.
  • When it came to exporting, I had to erase the parts I wasn't going to export.
    
    So, let's say my canvas includes: environment, puppet stage, and characters. 
    
    If I only want to export the puppet stage, I would erase everything but the stage from the canvas; save the sketch as a separate file; then export to .fbx.
    
    I did a lot of these so that each object could have its own .tilt file and could be easily reloaded, but it was really important for them to all live in the same canvas together too so I could visually design the relationships of all the components.
• Wrote a script
• Recorded audio for the story. Made audio edits and splitting into individual clips
• Learned Adobe Audition
• We achieved a basic puppet recording!

Challenges

• Setting up stage and puppets
  • Required a lot of revisions to get the puppets to record
• The story wasn't working as this project evolved. There became holes in the story that weren't making the most sense. 
• While collaborating in VR in BigScreen, we couldn't tell if we were confused, happy, etc — it's hard to determine an accurate reaction of content review when there are no facial expressions in VR
• It was really hard to act out our story. It sounds easy but was really hard. We're not theater ppl and that showed. 
• For the recording: simultaneous blocking and audio capture was tricky. 

day 3: Sunday

OBJECTIVES + Accomplishments

• Painted out skybox, characters
• Audio split up lines
• Acted out Scene One in VR and live/real scale
• Got much longer recording run-thru done
• We figured out how to extend the length that we could record
• Revised our plan at 4PM:
  • to create an interactive play space with the models created
  • scrapped the story
  • Continued with character and stage painting
  • Added props into the scene to play with basic physics
  • Looked into GIF recording; cam will save GIFs!

Challenges

• It was tough to record with one person in and one person out of VR. Blocking was tough, and general choreography! 
• Found that Unity lacked effective UI for animation
• Felt really challenged / walled when we realized our animation limitations and abilities. We took a long break to decide if we wanted to continue, to scrap it, or revise. We decided to simplify and revise our plan at 4PM.