Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
8d49297
updated PartA
eigenValue7 Aug 31, 2025
fe9a879
updated name
eigenValue7 Aug 31, 2025
cfc54ec
updated images
eigenValue7 Aug 31, 2025
61bdb54
updated readme
eigenValue7 Aug 31, 2025
180e6f7
updated readme
eigenValue7 Aug 31, 2025
dd569a7
updated the part1
eigenValue7 Sep 3, 2025
a5e201c
edit format
eigenValue7 Sep 3, 2025
07f01b4
edit format
eigenValue7 Sep 3, 2025
e682dd2
edit format
eigenValue7 Sep 3, 2025
a316e34
edit the format
eigenValue7 Sep 3, 2025
d616bd1
finish section2
eigenValue7 Sep 7, 2025
7c20f63
Merge branch 'Fall2025' into Fall2025
eigenValue7 Sep 10, 2025
ca46f11
updated to show the lionking
eigenValue7 Sep 17, 2025
c09c827
feat added wicked
eigenValue7 Sep 17, 2025
47c3e29
modified the words
eigenValue7 Sep 17, 2025
53b806d
wip music on pi
Alubeto Sep 20, 2025
d0744e9
added new file screen curtain
eigenValue7 Sep 20, 2025
f3ce83f
merge
eigenValue7 Sep 20, 2025
25717ba
updated readme
eigenValue7 Sep 20, 2025
94d2ee2
updated readme
eigenValue7 Sep 20, 2025
bcf81b9
adding double click upvol
Alubeto Sep 20, 2025
c2509ff
updated readme
eigenValue7 Sep 20, 2025
b39b5a2
volume working full
Alubeto Sep 20, 2025
4a76ed7
playing wrong song but things are working
Alubeto Sep 21, 2025
e2ad7d3
cleaning up code
Alubeto Sep 21, 2025
32f4362
adding videos to read me & fixing audio file name
Alubeto Sep 21, 2025
07b4875
adding better name for link
Alubeto Sep 21, 2025
d9c9336
making it play different music if the show lottery is open
Alubeto Sep 21, 2025
798c199
Update README.md
eigenValue7 Sep 22, 2025
48b74a6
Merge branch 'Fall2025' into Fall2025
eigenValue7 Sep 24, 2025
ab200dc
Merge branch 'IRL-CT:Fall2025' into Fall2025
eigenValue7 Sep 24, 2025
56459f7
updated
eigenValue7 Sep 24, 2025
355aa57
added the phone number
eigenValue7 Sep 26, 2025
a6812c3
updated
eigenValue7 Sep 28, 2025
50cad4e
Update README.md to add description for dialogue & other storyboard
celestialskys Sep 29, 2025
d370a7e
added
eigenValue7 Sep 29, 2025
f3a48ff
Merge branch 'Fall2025' of https://github.com/eigenValue7/Interactive…
eigenValue7 Sep 29, 2025
1fb1d67
added ollama test
eigenValue7 Sep 29, 2025
a0c1181
added the diagram
eigenValue7 Sep 30, 2025
8d69f28
added the chat test
eigenValue7 Oct 2, 2025
a0e9604
added fridgly
eigenValue7 Oct 2, 2025
09e27f4
modified readme
eigenValue7 Oct 2, 2025
0569eb3
updated readme
eigenValue7 Oct 2, 2025
0052328
updated images to the right direction
eigenValue7 Oct 2, 2025
41879af
updated image
eigenValue7 Oct 2, 2025
e60e303
updated image
eigenValue7 Oct 2, 2025
99e92d7
updated readme
eigenValue7 Oct 2, 2025
55853e0
edit readme
eigenValue7 Oct 5, 2025
8f41346
edit readme
eigenValue7 Oct 5, 2025
a5c23d6
updated lab3
eigenValue7 Oct 6, 2025
23c5af2
Update readme
eigenValue7 Oct 6, 2025
6643542
update image
eigenValue7 Oct 6, 2025
7f2fa37
update
eigenValue7 Oct 6, 2025
87c2dc9
Merge branch 'IRL-CT:Fall2025' into Fall2025
eigenValue7 Oct 10, 2025
b420343
readme
eigenValue7 Oct 13, 2025
c62fd54
udpate
eigenValue7 Oct 13, 2025
b74e184
updated
eigenValue7 Oct 15, 2025
6b0a974
updated lab4
eigenValue7 Oct 15, 2025
155a816
Updated lab4
eigenValue7 Oct 15, 2025
93db8af
update
eigenValue7 Oct 16, 2025
68d9a77
updated
eigenValue7 Oct 16, 2025
18c7488
updated
eigenValue7 Oct 16, 2025
1b440d0
merging in sound changes
Alubeto Oct 16, 2025
0b81bd6
adding sound folder and files
Alubeto Oct 16, 2025
df6eaf6
wip sounds do work but need to change with joystick
Alubeto Oct 16, 2025
da1db72
drawing sound working
Alubeto Oct 16, 2025
76d5f37
fixing audio player
Alubeto Oct 16, 2025
40d7e17
updated gitignore
eigenValue7 Oct 16, 2025
c1f46e5
updated
eigenValue7 Oct 19, 2025
dacfd04
updated
eigenValue7 Oct 19, 2025
eb164ef
updated readme
eigenValue7 Oct 19, 2025
c62f55d
updated readme
eigenValue7 Oct 19, 2025
bdcd265
updated readme
eigenValue7 Oct 19, 2025
3c7b3bc
updating ReadMe for images svgs and details
Alubeto Oct 20, 2025
ad9309e
adding other sketches for prototype
Alubeto Oct 20, 2025
a713334
adding other demo video
Alubeto Oct 20, 2025
303c11f
Merge branch 'IRL-CT:Fall2025' into Fall2025
eigenValue7 Oct 20, 2025
d9e435a
added cry analyzer
eigenValue7 Oct 28, 2025
324ca98
update cry_analyzer
eigenValue7 Oct 28, 2025
918f8dd
updated to detech laugh/cry
eigenValue7 Oct 29, 2025
c8dc486
updated to detect laugh and cry
eigenValue7 Oct 29, 2025
87a74f3
udpated readme
eigenValue7 Oct 29, 2025
02e353a
updating readme p1
Alubeto Nov 2, 2025
3146b7a
updating readme p2
Alubeto Nov 2, 2025
f3f0078
hiding long part A instructions
Alubeto Nov 2, 2025
36965ab
updated readme
eigenValue7 Nov 3, 2025
608c98b
updated readme
eigenValue7 Nov 3, 2025
4bf7beb
Merge branch 'Fall2025' into Fall2025
eigenValue7 Nov 3, 2025
8798800
led color analyzer
Alubeto Nov 4, 2025
0975979
adding imgage of light up and video leds
Alubeto Nov 4, 2025
54b9562
updating read me for leds
Alubeto Nov 4, 2025
de06a33
Merge branch 'IRL-CT:Fall2025' into Fall2025
eigenValue7 Nov 5, 2025
d1373e9
updated game
eigenValue7 Nov 6, 2025
10c910f
joystick code
Alubeto Nov 6, 2025
1b8ecc3
renaming joystick script
Alubeto Nov 6, 2025
d24aa43
moving joystick code to game folder
Alubeto Nov 6, 2025
1490140
updated readme + game_server
eigenValue7 Nov 6, 2025
feb75c0
working joysticks game
celestialskys Nov 7, 2025
944be0c
fixing indent
celestialskys Nov 7, 2025
48132c8
updated readme
eigenValue7 Nov 10, 2025
9a75cb7
updated
eigenValue7 Nov 10, 2025
0045329
udpated
eigenValue7 Nov 10, 2025
ff0e354
updated requirement
eigenValue7 Nov 10, 2025
dfca1e7
Updated readme
eigenValue7 Nov 10, 2025
016f05b
updated readme
eigenValue7 Nov 11, 2025
58a3931
updated with hapticmotor
eigenValue7 Nov 17, 2025
92b3978
updated readme
celestialskys Nov 17, 2025
dbaeacd
Merge branch 'Fall2025' of https://github.com/eigenValue7/Interactive…
eigenValue7 Nov 17, 2025
60cabd0
updates for pressure sensor and sound
celestialskys Dec 3, 2025
21e36f8
wip sensors working
celestialskys Dec 5, 2025
80b5773
wip all sensors
celestialskys Dec 5, 2025
5720f3e
wip the vibes and led are working next other sensors & sounds
celestialskys Dec 5, 2025
ad0c8aa
wip purring
celestialskys Dec 5, 2025
3a61664
wip sounds2
celestialskys Dec 5, 2025
03f0bff
wip sound
celestialskys Dec 5, 2025
6d6f7c2
wip touch sound vibes
celestialskys Dec 5, 2025
8dbec4e
wip working but sensors are being activated on top of each other;
celestialskys Dec 5, 2025
76b2a5b
sound pressure sensor working w lights
celestialskys Dec 5, 2025
ddb1ba3
works but few little details to fix
celestialskys Dec 5, 2025
71d5932
vibing with lights on pressure sensor
celestialskys Dec 6, 2025
6b22047
everything is working
celestialskys Dec 6, 2025
d592c15
updated readme
eigenValue7 Dec 9, 2025
d04b0ca
updated
eigenValue7 Dec 9, 2025
b3f89ce
updated readme
eigenValue7 Dec 9, 2025
87af232
updated readme
eigenValue7 Dec 9, 2025
c762366
updated
eigenValue7 Dec 9, 2025
15112e9
edit photo
eigenValue7 Dec 9, 2025
5425272
updated paragraph
eigenValue7 Dec 9, 2025
47597e6
added videos of other person testing it
celestialskyz Dec 9, 2025
31d6f55
adding response from event
celestialskyz Dec 9, 2025
2be710c
adding our names to the top
celestialskyz Dec 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,6 @@ venv
venv*
__*
.venv*
.venv/
.venv/
.*venv*
*.csv
330 changes: 330 additions & 0 deletions Final_project/README.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,330 @@
<!-- Put all of this information on your project Github, and submit the link to the Github.

1. Project plan: Big idea, timeline, parts needed, fall-back plan. (Can be same as previous turn-in, but updated if you changed your plan)

2. Functioning project: The finished project should be a device, system, interface, etc. that people can interact with.

3. Documentation of design process

4. Archive of all code, design patterns, etc. used in the final design. (As with labs, the standard should be that the documentation would allow you to recreate your project if you woke up with amnesia.)

5. Video of someone using your project -->
Group: Nana Takada (nt388) & Lianne (Celeste) Bisch (lb854)

# Project Plan:

## Big Idea:
A soothing plushie that replaces harmful stims (e.g., head banging) with comforting light, sound, and touch feedback to support emotional regulation.
- Input: When/where/how the user touches the plush
- Output: Soothing lights pattern, haptic feedback, and/or voice
We will test and update the input/output based on the user feedbacks

## Materials we need
- Raspberry Pi 5
- Woven Conductive Fabric - 20cm square
- Stainless Thin Conductive Thread - 2 ply - 23 meter/76 ft
- Adafruit DRV2605L Haptic Motor Controller - STEMMA QT / Qwiic
- Vibrating Mini Motor Disc
- 1N4001 power diode or a level converter chip 74AHCT125 (USB microphone & Mini speaker (Logitech))

## Testing plan
- Functional Testing
- Haptic Response
- LED lightin
- Microphone Input & Response
- Usability Testing with two people
- Ease-of-use
- Feedback preference
- Emotional Effectiveness

## Fallback Plan
For all types of feedback, we can conduct a Wizard-of-Oz study to control when and how the feedback is delivered. For example, for haptic feedback, the wizard can manually trigger the vibration when the user touches the plushie
Choose 1 type of feedback to focus on
- haptics fail - add small manual massager
- color change problems- use reversible sequin fabric
- audio issues - hard code some soothing phrases for responses


# Functioning project

**MewoPow**

A soothing smart plushie that provides comforting light, sound, and haptic feedback to support emotional regulation.

<img src="img/final_pic.jpg" width=200>

[Demo video](https://drive.google.com/file/d/1ILqqUDU1AAwQOzUwiy2DEs0eBlaDfm7g/view?usp=drive_link)

- **LED lights** – Calming light patterns
- **Haptic feedback** – Gentle vibrations
- **Audio responses** – Meows and purring sounds

### Interaction Design

| Interaction | Location | Detection | Response |
|-------------|----------|-----------|----------|
| **Petting** | Head (conductive fabric) | Capacitive touch sensor | 8-sec purring mode: looping purr sound + slow pastel color transitions (purple → lavender → pink → peach) + gentle vibration |
| **Patting** | Back (conductive fabric) | Capacitive touch sensor | 7-sec purring mode: meow sound + orange flash + pastel color transitions + gentle vibration |
| **Quick Tap** | Body (pressure sensor) | Press < 0.3 sec | Short meow + sparkle LED effect + burst vibration |
| **Squeeze** | Body (pressure sensor) | Press 0.3–1 sec | Meow + yellow chase LED pattern + vibration |
| **Long Hug** | Body (pressure sensor) | Press > 1 sec | Toggles **breathing mode** (4-7-8 pattern): blue↔white LEDs + rhythmic vibration to guide calm breathing |

#### Breathing Mode (4-7-8 Pattern)

When activated by a long hug, the plushie guides the user through a calming breathing exercise:

- **Inhale (4 counts):** LEDs gradually brighten, vibration increases
- **Hold (7 counts):** Full brightness with gentle pulse
- **Exhale (8 counts):** LEDs slowly dim, vibration decreases

# Documentation of Process

## 0. Our first ideation

We created storyboards envisioning how users would interact with the plushie at home—hugging, petting, or simply looking at it for comfort.

<img src="img/storyboard.jpg" width=300>

We explored different areas to incorporate lighting: belly, clothing, tail, and cheeks.

<img src="img/design.jpg" width=300>

## 1. Test with Haptic Motor and Touch System

We tested haptic motors and touch sensors.

### Haptic Motor

We tested the vibration motor using the [haptic test script](test/test_inhale.py), initially with the Adafruit DRV2605L haptic motor controller.

[Test Haptic Motor](https://drive.google.com/file/d/1jeg4uBKiy9n-tyOyDO9NiWyAg44exAlF/view?usp=drive_link)

### Conductive Fabric + Capacitive Sensor

We created a simple prototype plushie with conductive fabric on top, connected via conductive thread to a capacitive touch sensor.

<img src="img/initial_pic.jpg" width="300">

[Touch sensor test code](test/touch-sensor-test.py)

[Test Touch Sensor](https://drive.google.com/file/d/1ToM1HJ9TXLMcaa0mjYVwpd-h35VXzcin/view?usp=drive_link)

## 2. Ask for Feedback on our First Plushy

As we finished prototyping the intial version of the plushy, we asked 2 Cornell Tech students, Nana's sister, and her husband to give us the feedback on the size and sensors we were thinking to use.

Some intial feedbacks we got:
- They liked the haptic motor but they thoght it was too strong and not very steady
- They liked the size of the plushy but could be bigger but not smaller. It's a good size to hug
- One of them wants a bit more heavy. It's too light
- Everyone thought it's a bit too little of the interaction that there is no sound or other things to give us feedback. They want a bit more inteaction.

## 3. Add Additional Inputs + Outputs

Given the feedback, we decided to add more sensros as well as output forms.

### Pressure Sensor

We tested with the [pressure sensor test code](test/pressure-test2.py). Without an analog to digital converter chip, we chose to measure hug pressure based on the duration of pressure the sensor receieved, shorter being a tap, medium length being a squeeze and longer being a hug.

[Test Pressure Sensor](https://drive.google.com/file/d/1hq4O5mBmDS_iIKmoONX1ANE9k0dyZU5d/view?usp=drive_link)

### Led Ring

We tested with the [led ring](test/led_test.py)
For LED, we learned to run the led ring, it has to be connected with GPIO12.

### Speaker

For speaker, we just tested with
```bash
aplay lookdave/wav
```

## 4. Test with All Sensors at the Same Time

After confirming that each component worked successfully on its own, we tested multiple sensors running at the same time.
At this stage, we began encountering issues.

First, we discovered a compatibility problem between the haptic motor controller and the capacitive touch sensor.
Both devices communicate over I²C, and when we attempted to use them simultaneously, the system frequently threw an error indicating that address 0x5A could not be found.

To resolve this, we decided to stop using the haptic motor controller and instead connect the haptic motor directly to the Raspberry Pi through the breadboard.

We also attempted to use a Qwiic SHIM. However, despite soldering the parts together, the soldering pads were extremely small and difficult to work with. As a result, we were not able to create a reliable connection and ultimately did not use the Qwiic setup.

[Test All Sensors](https://drive.google.com/file/d/1V0iHdGeFqw7xdqrPQZ9Vq-6pxtXVIfxY/view?usp=drive_link)

## 5. Make the Outer Look

We increased the overall size to accommodate components and improve comfort based on user feedback.

<img src="img/final_pic.jpg" width=300>

## 6. Final Code for Interaction

[Final Code](master.py)


There are two main modes for this code.

**Breathing Mode**
Activated by a long hug (>1 second press). Implements the 4-7-8 breathing technique used in anxiety management:
- LEDs transition from blue to white following the breath cycle
- Vibration intensity syncs with the breathing pattern
- Stays active until another long hug toggles it off

**Purring Mode**
Activated by touching the conductive fabric (head or back). Runs for a set duration (7-8 seconds):
- LEDs slowly cycle through calming pastel colors (purple → lavender → pink → peach)
- Vibration oscillates to mimic a cat's purr
- Looping audio plays until the timer ends

**Twinkle In Eye**
- Singular white LED slightly right of the right eye which completed a circuit using a button battery when pressed.

```mermaid
flowchart TD
subgraph Inputs["👆 INPUTS"]
HEAD["🐱 Head Touch(Conductive Fabric)"]
BACK["🐱 Back Touch(Conductive Fabric)"]
PRESSURE["💪 Pressure Sensor(Body)"]
end

subgraph Detection["⚙️ DETECTION"]
CAP["Capacitive TouchSensor (MPR121)"]
FSR["Force SensitiveResistor"]
end

subgraph Processing["🧠 PROCESSING (Raspberry Pi)"]
TOUCH_LOGIC["Touch Logic"]
PRESSURE_LOGIC["Pressure Logic"]

subgraph Modes["Active Modes"]
PURR_MODE["😺 Purring Mode(7-8 sec timer)"]
BREATH_MODE["🌬️ Breathing Mode(4-7-8 pattern)"]
end
end

subgraph Outputs["✨ OUTPUTS"]
LED["💡 LED Ring(24 NeoPixels)"]
MOTOR["📳 Vibration Motor"]
SPEAKER["🔊 Speaker"]
end

HEAD --> CAP
BACK --> CAP
PRESSURE --> FSR

CAP --> TOUCH_LOGIC
FSR --> PRESSURE_LOGIC

TOUCH_LOGIC --> PURR_MODE
PRESSURE_LOGIC -->|"> 1 sec"| BREATH_MODE
PRESSURE_LOGIC -->|"< 1 sec"| LED
PRESSURE_LOGIC -->|"< 1 sec"| MOTOR
PRESSURE_LOGIC -->|"< 1 sec"| SPEAKER

PURR_MODE --> LED
PURR_MODE --> MOTOR
PURR_MODE --> SPEAKER

BREATH_MODE --> LED
BREATH_MODE --> MOTOR

style HEAD fill:#FFB6C1,stroke:#FF69B4
style BACK fill:#FFB6C1,stroke:#FF69B4
style PRESSURE fill:#87CEEB,stroke:#4169E1
style PURR_MODE fill:#DDA0DD,stroke:#9932CC
style BREATH_MODE fill:#98FB98,stroke:#228B22
style LED fill:#FFD700,stroke:#FFA500
style MOTOR fill:#FFA07A,stroke:#FF6347
style SPEAKER fill:#87CEFA,stroke:#1E90FF
```

#### Pressure Detection Flow

```mermaid
flowchart LR
PRESS["Press Detected"] --> DURATION{"Duration?"}

DURATION -->|"< 0.3s"| TAP["⚡ TAP"]
DURATION -->|"0.3s - 1s"| SQUEEZE["🤏 SQUEEZE"]
DURATION -->|"> 1s"| HUG["🤗 HUG"]

TAP --> TAP_OUT["Sparkle LEDs+ Burst Vibration+ Short Meow"]
SQUEEZE --> SQUEEZE_OUT["Chase LEDs+ Vibration+ Meow"]
HUG --> HUG_OUT["Toggle BreathingMode (4-7-8)"]

style TAP fill:#FFD700,stroke:#FFA500
style SQUEEZE fill:#FFA500,stroke:#FF8C00
style HUG fill:#FF6347,stroke:#DC143C
```

## Some feedbacks from Demo Day
People liked the weight the pi and other components gave the plushie because they made him more huggable and felt more real. They liked the multiple sound options and lights. We found that we should let the other sensors time out the hug more quickly and that we could better manage overlapping inputs since when hugged many would touch the back conductive fabric and squeeze the pressure sensor causing some overlap in response and a longer cool down period where the pressure sensor would stop. One person recommended adding a gyroscope for more interaction so the plush could tell if it was being held upside down or shaken and to provide an accompanying yelp or unhappy sound/output. They appreciated that there were no exposed wires needed and that it could be run without being plugged into anything. Overall they thought it was cute, portable and very cuddly. They liked the sound and said it would be a great alternative to a pet for those with allergies or anyone who doesnt want the responsibility of an actual pet.

# Final Code/Setup

## Code

### How to run
```bash
source .venv/bin/activate
pip install -r requirements.txt
python master.py
```

If the code does not work or give an error, please try to run below:

```bash
cd test

# Test led
python led_test.py

# Test speaker
aplay lookdave.wav

#Test pressure sensor
python pressure-test2.py

#Test haptic motor
python test_inhale.py

#Test capactive sensor
python touch-sensor-test.py

#if this does not work, then ru
sudo i2cdetect -y 1

#This should show 0x5A. If this does not print 0x5A, double check the hardware setup.
```

## Hardware

### Hardware We used

| Component | Purpose |
|-----------|---------|
| Raspberry Pi 5 | Main controller |
| Woven Conductive Fabric (20cm²) | Touch-sensitive surface |
| Stainless Conductive Thread (2-ply, 23m) | Connecting fabric to sensors |
| MPR121 Capacitive Touch Sensor | Detecting light touch/petting |
| Pressure Sensor | Detecting hugs/squeezes |
| Vibrating Mini Motor Disc | Haptic feedback |
| NeoPixel LED Ring | Visual feedback |
| Mini Speaker | Audio output (meows, purring) |
| 1N4001 Power Diode | Circuit protection |

### How to set
<img src="img/1.jpg" width=400>
<img src="img/2.jpg" width=400>
<img src="img/3.jpg" width=400>
<img src="img/4.jpg" width=400>
<img src="img/5.jpg" width=400>

# Video of External User's Interaction

[FriendTestingPlushp1](https://drive.google.com/file/d/1ZDuw2JSaymoYfkVlbT1KC0A8QLrDWdW9/view?usp=sharing)
[FriendTestingPlushp2](https://drive.google.com/file/d/1IiOh0pzbwk9lNgIRKDTvq4dx6eMaAt9-/view?usp=sharing)
Loading