Assignment 1: Spatially Augmented Reality

Assignment 01
Create an spatial augmented reality experience:
Find an element in a room or wall and augment it.
Give it life using data, user interaction or any other means.
Remember the 3 characteristics of AR/MR.
   1. combines real and virtual
     2. interactive in real time
     3. registered in 3Dspace 

Jess and I worked on this assignment together.
We knew we wanted to use some object on the floor as our projection surface, so after much scouring and thinking, we decided to use one of the many dress forms.

We decided to use Blob Detection for Processing to track a particular color using computer vision as our interactive input. The idea was to use multiple colors and attach images of different organs to the colors and pin them to the dress form.

We also wanted to funnel the Processing sketch via Syphon to Madmapper, which is the first time we had used either Syphon or Madmapper. Here are some experiments:

Unfortunately, since we were using video in processing, we couldn’t figure out how to adapt our code properly so that the computer vision aspect would show up in madmapper. All we were able to get was the video feed via Syphon.

Therefore, we just decided to fullscreen the processing sketch and adjust the projector. We also just used a single color as a proof of concept.

While we didn’t execute the full vision, it was a good exercise to jump back into some processing sketches and also a decent introduction to MadMapper, which is awesome!


HSF Final: When We Die

The ability to contemplate our finite existence is a fundamental part of the human experience, but we rarely reflect on it in our daily lives until it is often too late or under traumatic circumstances. This is particularly apparent in Western culture where death has been largely removed from modern life and when apparent, is an intensely medical process.

Granted, it is uncomfortable to acknowledge our eventual nonexistence, our vulnerability and fragility, and the idea that all that we have in this world – material and immaterial – will be lost. However, if we are able to confront the idea of our mortality more often, perhaps it can be motivating, even clarifying. It can help us focus on the things that are truly important to us, as well as prepare for when the inevitable occurs.

When We Die is a virtual reality experience that guides users through the process of contemplating their own mortality and presents points of view that may not previously have been considered.

The experience is divided into two parts:

  Part 1: A guided mediation asks users to open themselves to the idea of their nonexistence using the analogy of a tree cycling through seasons. The imagery consists of an open field with a single tree in the foreground and takes on the style of a color film negative. In other words, a world that is both familiar and dissimilar to their own. The world cycles from night, into day, and then back into night. As the meditation brings the user back to their awareness, the world transitions to scene 2.

Cycles – Part 1

Stephanie’s meditation (voice of Richard Lapham)

Part 2: The user is transported to a colorful night sky above and  a calm and reflective ocean of water below. In the sky are glowing, rotating shapes. An audio cue informs the users that they may select from different audio clips by holding their gaze momentarily over the shapes. The audio clips are snippets from our interviews with our experts, Stephanie Hope, RN, BSN & Gayatri Devi, MD.

Devi Audio Clips

Hope Audio Clips

During the show, we set up a white tent in the corner of the room and lined the floor with pillows and soft lights. Given the madness of the ITP show, it was a relatively comfortable and serene space.

Joe Mango
Joe Mango
Winslow TPIII
Winslow TPIII
Stephanie Hope et al.
Stephanie Hope et al.











We were fortunate in that both Stephanie and Dr. Devi were able to come and try the experience. Happily, both of them expressed how impressed they were and thought it was really well done. Stephanie watched it twice and told us that it felt really validating from a professional standpoint, which is perhaps the highest praise we could have received throughout the evening.

Overall, we had some really great conversations with people over the course of the 2-day show. The vast majority of people described the experience as “beautiful” and “moving,” even “serene.” We also had a handful of people who chose not to partake due to recent circumstances in their own life or who felt really uncomfortable afterwards, for the same reason. Fortunately, they were very explicit with us about their criticisms. For example, one woman with such recent experiences told us that the fact that the trees and grass weren’t realistic in color was disturbing to her. She wanted the contrast of a “living” environment when listening to such a narrative, which is something to consider.

We also had great conversations with many health care professionals who work with dying patients. Many of them (and others) implied that this might be an interesting piece to show to terminally ill patients, not dissimilar to what the Psilocybin Project does.

Somewhat hilariously and serendipitously, one man who had actually done a project about dying in VR recently also went through the experience and thoroughly enjoyed it. He is listed in this article that was published barely a month ago, so it seems it’s a hot topic. He has actually already reached out and was interested in chatting further.

Importantly, we gathered a list of a lot of technical details that we need to iron out, including:
* Make time restraints in last scene known
* Move the shapes down so they are all in people’s immediate field of vision
* Disable already played audio clips
* Work on transition scenes

In summary, the show was immensely useful from a user-testing standpoint both technically and conceptually. We weren’t sure how something like this would be received and though we had a few negative reactions, the majority were overwhelming positive and emotionally touching. People seemed really excited by the idea of meditation in VR and thought the technology paired incredibly well with the idea.

I think we would all agree that this is something we would like to continue to work on in the future. We are working on the website and hope to reach out to some of the people from the show to continue to gain feedback about future iterations.





Immersive Listening: Midterm Proposal

The midterm will be a 360 video with spatial audio for which sound design is at the center of the concept. While the visuals are an inseparable part of a 360 that give context, you will be evaluated based on the sound design and not the visual components in order to ensure that the audio is your main focus. It must be published in away that allows for viewing on a VR headset of your choice.

For my midterm, abstractly, I am interested in creating a very simple visual that really focuses on the details of sound design. I will definitely use the Theta as I’ve had some experience with it, but have not yet recorded and edited video with it.

Searching for some inspiration online was not fruitful in the sense that there weren’t too many creative applications of 360 sound/video — they are all quite literal. However, it did confirm that I am interested in creating something more abstract

It’s important to me to have a space that is interesting in all directions, whether naturally occurring or curated.

Conceptually, I am interested in creating something around the idea of disembodiment, especially considering the POV afforded by this technology. I want to use this as a way to explore the body from outside the body. This would obviously be my own interpretation, but what does the my body sound like? How do the parts relate to whole? How does the body relate to the mind?

The vision for this is as follows: a room (a tented microstudio, perhaps) filled with white strips of fabric.e240f878a82379bdca47049af4dbf99f



At once, several body parts would appear from cracks or slits in the fabric and move in slow, but sustained ways. Some the body parts would come and go, but nearing the end of the video, a person would be suspended at the top of the tent. The viewer would be directed there with an overhead sound/narrative.

Obviously, the script needs some fleshing out, but I’m committed to working on this.


Final Project Idea: Death

Present your group’s chosen project (1) Frame the problem (WHY), (2) What’s the story / idea (3) Inspiration / References (4) Next steps

For my final project, I am working with Leslie Ruckman and Paula Ceballos Delgado.

When we sat down to chat about ideas, Paula explained that she had just lost a past co-worker in Chicago due to a car crash and had consequently been spending the past week thinking about death and about how we rarely consider our own mortality. From here commenced a discussion about different cultural views surrounding death and how western society tends to ignore its existence in our daily life.

I expressed that I had been to a lecture a couple years back by a researcher at NYU who was involved with a study that used psychedelic drugs for therapeutic treatment with terminal cancer patients. During the lecture, he cited how many of these patients would come for a single session and that they expressed having ‘spiritual’ experiences during the sessions. Many of them cited feelings of relief and said they felt as though they had come to terms on some level with their impending death.

What interested all of us about this story wasn’t so much the aspect of psychedelics, but the unique cross-section of individuals who were involved with this study. These are people who are facing their own mortality and given their willingness to try a psychedelic session, seem to be open to exploring alternative narratives to typical western ways of dying.

From there, we knew we wanted to explore the concept of Dying in a Western World. What does that mean? Why do other cultures have much more intimate relationships with death? What happens when you must face death head on? How can we re-write our own cultural associations?

Our story as of now is more about presenting narratives and/or presenting different ways in which people face death. Much of how our story is formed with hinge on with whom we can speak.

Right now we are entertaining the idea of overlaying a narrative in VR and creating more abstract visuals that pertain to the story.

I remembered a friend of mine from a “past life” went to NYU Nursing school and ended up working with this team of researchers, so I reached out to her. She now lives and practices in Connecticut as a hospice nurse and we are slated to have a discussion with her on Sunday morning.

She is also trying to contact members of this team to see if they would be willing to have a discussion, though we have reached out on our own.

Listening Project
New Yorker
Dying in VR
Tibetan Book of Dying

Our next steps are to speak with some people and do some more research to really solidify what the project looks like. We are keeping it open in order to make sure that we create a project that is relevant and gives the concept the respect it deserves.

Midterm Concept & Demo: 100 ppb

100 ppb

Since my last iteration of this project, I realized that my thinking became too large, as did my subject matter. I decided to narrow in and focus on the originator of the Citizen Spring project, which was the water crisis in Flint, MI.

The crisis is one of those issues of which people are aware, but which the details are unknown or relatively obscured in rogue papers and news sites. I haven’t lived in Michigan for years, but I did grow up an hour from Flint and have been to the place that is only a sound bite in most people’s minds.

I knew that I wanted to create something for the web, so that it was the most accessible to everyone since truly every person in the United States drinks water, most likely from the tap.

My idea was to create a 3 part project: An informative story that used the voices from those who lived and are living through this crisis, an amalgamation of accurate information about our country’s water infrastructure (ie. how does lead happen in the first place), and a tool for people to help take control of their own water quality.

After presenting the idea, I received a number of good pieces of feedback that I took to heart.
1) I need to figure who this project is for? My original intention was to make something for the people of Flint, but I realize now that that is not clear. It almost seems like something aimed toward a more general, affluent user group.

2) Another interesting comment was that it seems like I should make something that extends beyond the simple interaction of a mouse scrolling. In other words, if felt like it lacked interactivity. I whole heartedly agree and I don’t think the demo clearly demonstrates the interaction that I intended. Perhaps that interactivity comes from user’s being able to submit their own stories?

3) My last and favorite comment was, if I intend this to be for the people of Flint, there is something powerful in simply asking, as a designer and person with access to new technologies, what is needed, instead of trying to guess. I love this idea, but I still feel reticent to reach out without feeling like I have more to offer at this point.

I’m not sure if this is a path on which I want to continue yet. While I feel it is important and current, I’m not confident that I’m the one to tackle it at this point. It seems so large and important and I would love to have something tangible at the end of this class.

Demo of Website

100pb Demo from Dana Abrassart on Vimeo.

Week 3: Form 2

Write a sketch that generates 2 shapes on a page. The first shape should be your design of the word “wet”. The second shape should be your design of the word “sharp”. Use only black and white. You have to use either Rune.Polygon or Rune.Path objects to draw the two shapes, and please try to do a design that is better done in code than in hand. Print at AMS.









Week 2: Form 1

This week, I want you to design a simple book cover for the book The Martian. You can only use basic shapes (rectangle, triangle, ellipse, line) and black/white. Think about how you can use the basic relationships (shape, position, dimension, rotation) to illustrate the point you’re trying to make.

I knew that I wanted to create an extremely simple and minimal design and let the placement of the objects speak for themselves. I used the relative orbits of the Earth and Mars around the sun. I wanted the Earth to be the first thing the eye would see (scanning from the left to right) and then Mars, but in full view.






Week 4: Concept Design Package

This Week
1. Create a User Persona
2. Start designing the project based on what you know about your users, goals, vision: A mix of spatial maps, sketches, etc. Must include a user journey. Use pre-during-post framework
3. Mood boards, Mock-ups, Wireframes, Sketches in response to what you identified above

Building upon last week, I knew that I wanted to keep on the water theme, but I wanted to expand and move beyond just lead in water. But in order to move beyond, I had to move backward: Where do lead and other potentially harmful water contaminants come from? When was our water and sanitation infrastructure laid? By whom? Where does our water come from? Who controls and regulates our water sources? Who should? Is clean water a human right?

It was the last 3 questions that really interested me because the answers to these seemed to be at the heart of the “What’s in your water?” issue. They also made reference to the future circumstances of water. I did a bit of research and ended up reading more about water control and privatization and the story I found was conflicted and complicated. As with most issues, there are advocates on most sides of the coin.

It left me confused and wondering what the reality of the situation is for those who live this. Perhaps there is a unanimous and resounding opinion or maybe experiences greatly differ depending on geographic location. What does privatization look like in New Jersey vs. Bolivia? How do local custom and cultural shape water consumption habits and would big business be sensitive to these aspects of life?

It was here that I realized my attempts to help solve a problem of epic proportions were beyond my current scope. Instead, I wanted answers to these questions and an interesting way to help engage others in these questions.

I wanted something that would use technology to help tell the story, but I didn’t want the technology to limit who can access the project, so I decided on the web, probably the most democratic of new technologies.


Our own Rebecca Rick’s animated rainfall chart : I could see animated graphics and data like this working well to tell the story
Quipu Project : Inspired by the “narration-less” style. It adds power, depth, and ownership
Virtual Water : Fantastic way to disseminate info in an engaging and visually appealing way. Simple and clean.

Proposed Project

BODIES of WATER ~ an interactive webdoc

Bodies of Water is a project to explore the effects of privatization of the water sector through presentation of personal narratives of those who live in such municipalities or “conflict zones” and an interactive dashboard to present technical information in a visually pleasing and interactive way. It is about giving a human face to those who are and may be subject to the effects of water and therefore, human life as a commodity.

* Explore privatization of water through personal narratives and daily blogs/vlogs.
* Each story and body will have a dashboard of information about water sources, quality, news. Short intro doc told through images, news, etc — not traditional narration.
* Page will also provide resources and ways to take local action. Can sign up

User Persona:



User Journey/Mockup:

Guerilla marketing on public faucets
Internet searches
Directed from associated sites (NRDC, UN, etc)





Advocacy and Action Links
Ability to contribute to the project – add your own story (blog, vlog) that is location based
Links to local municipality information

There is still quite a bit to design and figure out how to layout, but the general framework is coming along. I also think the story can be pinpointed further, but