Research + App Dev.
Reality Computing
for Urban Development
Collaborators
Trade Institute of Pitt.
Project RE_
Location Junction
Architecture:
UDBS students
Project Space
Non profit developers handle physical construction
Our Role
The Computing Team:
Reality computing for this project encompasses a constellation of technologies focused around capturing reality (laser scanning, photogrammetry), working with spatial data (CAD, physical modeling, simulation), and using data to interact with and influence the physical world (augmented / virtual reality, projector systems, 3d printing, robotics).
We are working towards applying these technologies to real world problems such as residential design, sustainability, and infrastructure monitoring. My role specifically works towards providing immersive experiences for the residents to involve them within the design process and engender trust within the community.
StoryMap + Research Compilation
Sources:
Pittsburgh Archive
Healthy Ride
US Census, etc
Tools:
ESRI ARCGis
D3.js
Gather relevant information and ARCGIS data sets to best assess site potential. Understand the life of past and current residents, and extrapolate future forecastings. Prepare to present relevant findings to developers, investors, residents, and backers.
In progress interactive data visualization and story map presentation below.
Previous iteration
Context Visualizer
Hardware:
Rico 360 Cam
LIDAR Faro
Software:
Rhino
SketchUp
Revvit
Autodesk Excap, etc
Build VR tool to allow architects and developers to see designs in context, better understand spatial, visual, and environmental factors. (sunlight through the neighborhood, size relationships, etc) Use and experiment with various technologies to refine a pipeline for generating VR environment.
360 photo in Unity Tutorial | Ricoh Theta 360 Cam app | Sketchup Texture Mapping tutorial | Lidar Raw Files | download sketchup for modeling! | sketchup site model from google | possible AWS web to adaptive unity...search CrossoVR on youtube
One of the East Liberty Development Managers walking through the neighborhood with the architect proposed changes. HTC Vive. Photo cred to Soonho Kwon
VR environment was put together in Unity where we finessed lighting, wind, and realism
Photogram and lidar houses weren't high enough quality, so I modeled them in SketchUp based off onsite photographs I took.
A lidar scan of the area provided a point cloud dataset that could map out terrain in detail using Autodesk Recap's ReMake. Unfortunately, the information was still too messy to use. Terrain elevation from Google isn't at a high enough density to get site specific info
Home Incubator Experience
Skills:
Critical thinking
Rapid prototyping
Software:
Unity
AR kit + core
Vuforia
A-Frame
Having figured out different capture technologies and 3D asset generation pipelines, we need to begin to consider why, when, and how they should be used with different technologies for a specific experience? To design for those interactions, we also have to design for the experience we want to give visitors. How do we best convey information? What do we convey and why?
Initial mobile incubator grant proposal to Ford.
Architects efforts towards affordable housing
Discussion with developers and community leader. Getting to know residents through interviews with direct feedback from neighborhood leaders.
Overarching experience
Everyone worked on making VR versions of their individual sites in the beginning, but after collective planning, we split up to deal with different aspects. Vikas was kind enough to gather everybody's contribution in an overview.
Onboarding with a chatbot
Worked with Vikas on an iteration of an onboarding and follow up experience. Documentation
- Why a Chatbot? The idea was a synthesis of multiple goals: on-boarding, data collection and managing incubator activities. Collecting data on individual buyer helps craft a personalized experience both within the chatbot(suggestions and financial consultation services) and during the visit to incubator. More importantly a chatbot allows us to present with a buyer 24x7, just in time. We paid close attention to crafting conversations which felt more humane instead of traditional bot tonality. We intended to represent chatbot as an extension of the ReCon team, almost making it feel like a member a chatting with the buyer. On the technical front, a chatbot can afford plethora of functionalities. Like in case of ReCon, chatbot affords conversations around personal data collection as well as built in AR Scanning capability for tangible takeaway.
- Understanding implications
— Chatbots can be effective communication tools, but we understand that there is a limit to how far this chatbot can assist potential buyers. Somewhere along the way, we have to have humans representing ReCon. Combining chatbot with the incubator makes a compelling case for a well balanced communication and outreach strategy for recon.
Conceptual Intentions- • How can we outreach to potential buyers in the most sustainable way?
- • How can we effectively onboard potential buyers without putting too much of a cognitive load?
- • How can we provide effective assistance to keep potential buyers motivated and invested in ReCon?
- • Can we tie multiple components of ReCon as an intervention to arrive at a sophisticated management solution?
Credit for conversational copy goes to Vikas
AR interaction explorations
I led our little team of 3 in exploring AR potential. 2 other students worked on projection mapping primers, another 2 on branding and interface. Our class' various VR sites were speed optimized by the rest.
Exploring the use of an interactive, variable marker
Playing with Vuforia reveals some computer vision limitations
Exploring the use of an interactive, variable marker
Augmentation through a 1 dimensional marker glitches out when it's too distorted by angle or glare. Either the computer vision needs to be supplemented with extended tracking and programs to extrapolate position based off previous frames and possible object recognition, or the system around which people would interact/use a 1 dimensional marker needs to be further delibrated.
Another alternative is a multidimensional marker. It would use the same computer vision Vuforia has to offer, but manipulate the virtual 3D augmentation according to the angles and visibility of certain markers in relation to each other.
Prototype of the AR takeaway brochure/card. Chosen house appears above its corresponding floor plan
Takeaway Proposal
Onsite Proposal
New grant proposal for mobile incubator to Ford.
Incubator parts designed with AR experience in mind
I forgot to ask someone to film me during the client demo. But I took what I learned in developing this experience and replicated it for a presentation of another one of my projects (designing a popup shop) where I wanted people to be able to walk around the scene and physical design at real scale. Experientially communicating and scale by tying digital models to the physical world with ARkit's visual-intertial odometry system.