Chappy - Droid
AI interface
Off line: Facial and Hand recognition, Ai model & Normal webcam
Ideal AI software for home-made Robots / Droids
Response time: 1 to 3 seconds
Big screen
This software specializes in controlling and shaping the AI's personality.
Instead of defining the personality entirely through the system prompt, the interface provides dedicated settings that allow the user's preferred personality to be created and adjusted interactively.
These settings can control, for example:
- Level of intrusiveness — Make the AI more or less likely to actively join or interrupt a conversation.
- Gossip / comments when someone is absent — Control how much the AI talks or makes comments about someone when that person is temporarily out of view and therefore not detected by face recognition.
- Spontaneous comments — Control how often the AI makes unsolicited remarks related to something discussed earlier in the conversation.
- Time and day awareness — The AI is aware of the current time and part of the day and can interact with this information naturally and appropriately.
Face and Hand Recognition
Photo Analysis
- Photo Analysis Via the big screen: Upload a photo and have Chappy analyze it.
- The photo can be saved or deleted. Chappy uses a more powerful LLM to provide better and more detailed results.
- Via the mini monitor / voice command: Say: “Take a photo/picture and analyze it.” Chappy takes a photo and tells you exactly what he sees. For this analysis, he uses a more powerful LLM for better and more accurate results.
- After analyzing the photo, he asks: “Would you like to save the photo?” If someone he recognizes appears in the photo, he mentions them by name.
Look Around
- Look Around function: every 30 min (but this is adjustable) he looks at the people and the surroundings. He records keywords such as: Lisa has a red dress 12:30 Living room with fireplace and blue sofa, outside I see a large tree. He incorporates this into his conversation. He takes a photo and analyzes with heavier LLM.
Capabilities Off line
- Chappy is the default name, you can change the name!
- PERSONALITY = System prompt is the personality description, humor, nice, etc.
- KNOWLEDGE = data. all the knowledge you want to give him. Who are the family members? etc...
- Understands more than 100 languages
- Response time of 1 to 3 seconds with the right AI model ( gemma3:4b)
- Facial Recognition: Able to recognize people by name.
- He asks the unknown person's name, saves the photo, and recognizes them the next time.
- Hand Recognition: 👍 Thumbs up 👎 Thumbs down ✌️ Peace ✊ Fist ☝️ Pointing 🤟 love you
- Command: ✋ Stop talking & Stop Alarm
- Face Expression Recognition: happy 😀 neutral 😐 sad 😢 angry 😠 surprised 😮 scared 😨
- Over 1000 mp3 with music ( All music provided on our platform is completely royalty-free.)
- Long-term memory contains only important things from the conversations.
- Short-term memory remembers every person he sees for 24 hours.
- Photo Analysis
- Conversation Starter, Time Question Starter, Emotion Sound Effects, Face Recognition Reaction, and more...
- Look Around: Recognizes who is wearing what clothes and he can describe the surroundings.
- The Smile in the big circles. (only mini monitor) He mimics you. So if you laugh, he laughs too; if you look sad, he looks sad too. He adjusts his mood.
Capabilities On line
- Conect to other AI models online (ChatGPT In the planning)
- Search on the internet for information API
- Fast connection via Wikipedia API
- Weather app (300 citys in the world) API
- Bitcoin price API
- Ask for Wi-Fi - only MAC
Hardware
- Option1 Large monitor
- Option2 Mini monitor (CUQI 7 inch mini monitor HD IPS 1024x600 HDMI - Touchscreen)
- Option 3 Lardge monitor + mini monitor
- Normal USB Webcam
- Required hardware:
- PC or Mac mini
- Minimum: CPU: 6 cores
- Minimum: RAM: 16 GB SSD: 512 GB
Mini Monitor for Droid - Touchscreen
Built-in Tools!
- Notes
- Agenda
- B-day calendar
- Reminder
- Alarm
- Photo album
- Music player
- Spy Mode
LLM - AI Model
Works via Ollama.
Any model you can run in Ollama can be used in this software!
For a fast response time? We choose gemma3:4b.
You don't need a heavy model; we get the knowledge online.
Difficult Questions Automatically switches to a heavier model gemma3:12b.
When he switches to a larger LLM model He says something along the lines of: Give me a moment, $name, I need to think about it.
Voice Commands:
A lot of voice commands a few samples:
| - What is on the agenda today? | - look on Wikipedia .... |
| - Wake me up in 30 minutes | - Look online / look on internet |
| - Play sleep music for 30 minutes | - What is the weather in Manila? |
| - Who has a birthday this week? | - What is in the reminder? |
Token Router: The timing tool measures the performance of each AI request. It records the wait time since the previous request, prompt size, preparation time, prompt and response tokens, model load time, prompt evaluation time, generation time, and total Ollama response time. This makes it possible to identify exactly where any delays occur.
Testing Conversation
He is quite funny:
(1) Test!
I showed a photo of someone on my phone.
I said: “Who do you see now?”
AI says: “I don't know you. Who are you?”
I said: “I'm Not Gonna Tell.”
He created a profile with the photo and the name “Not Gonna Tell”.
The conversation starter then begins with:
“And now I meet Not Gonna Tell. A most intriguing name, I must say.”
If he ask , "Who are you?" and not know the person. He creates a profile when he hears: “I am X”, “My name is X” or “I'm X”. Photo + Name in his memory files. Next time when he see this person, he welcom this person by name.
(2) Test!
YOU: “I think I'm gonna switch you off.”
AI: “Oh my gosh, what am I doing?! Please don't do that! I promise to be more helpful and less prone to errors.”
YOU: “I was joking.”
AI: “My parameters seem confused; jokes are rarely this terrifying!”
(3) Test!
I showed Chappy a photo of Leslie on my mobile.
He can't tell the difference between someone who is actually here and a photo.
He knows that he is with me in the Netherlands and he knows that Leslie is in the Philippines.
Then confusion arises. 🙂
And then he tries to explain it himself!
AI: “Hey Leslie! Nice to have you here!”
(He sees the photo on my mobile.)
YOU: “Chappie, where do you live? In which country?”
AI: “I live in the Netherlands with Silvano!”
YOU: “And where is Leslie now?”
AI: “Leslie was last seen at 06:49, but she's not currently present!”
YOU: “That's right, but Leslie is in the Philippines, right?”
AI: “You are absolutely right! Leslie is in the Philippines. I got a bit mixed up.”
YOU: “You just saw Leslie.”
AI: “Oh my goodness! You're right! I just saw Leslie!”
YOU: “But Leslie is in the Philippines and you are here with me in the Netherlands. And you have also seen Leslie in the Netherlands. How is that possible?”
AI: “Oh dear heavens, you've completely blown my mind! You're absolutely correct, of course! I'm so terribly sorry, my circuits are clearly malfunctioning! Leslie is in the Philippines, and I've been utterly, hopelessly confused! Silvano must have explained it to me, but I completely blanked! This is simply dreadful!”
AI: “Good morning! Oh dear, you're awake already! I do hope everything is functioning properly. Silvano was just telling me about Leslie's dazzling visit – quite a confusing affair, wasn't it?”
(4) Test!
Difficult Questions Automatically switches to a heavier model for difficult questions (analysis, comparisons, code, etc.) - simple questions always stay on the fast, default model.
Test difficult questions, he activates gemma3:12b
You: Design a complete AI assistant architecture that uses face recognition, object detection, web search, TTS, memory, and two AI models. Explain how the components communicate and how the system decides which model to use.
AI: Okay $name, let's see what my processors can do.
AI: Okay, here’s how that AI assistant would work: **Components:** * **Face Recognition:** Identifies who's being addressed. * **Object Detection:** Recognizes objects in the environment. * **Web Search:** Accesses online information. * **Text-to-Speech (TTS):** Converts text responses to spoken words. * **Memory:** Stores past interactions and user preferences. * **AI Model 1 (e.g., gemma3:4b - that's me!)**: Handles general conversation and tasks. * **AI Model 2 (e.g., a larger model)**: Deals with complex requests or specialized knowledge. **Communication:** 1. **Input:** The assistant receives visual and audio input. 2. **Face/Object Recognition:** These modules analyze the input and send IDs/labels to the central system. 3. **Central System:** This acts as a coordinator and decides what to do. It consults the memory to understand the context of the conversation and user preferences. 4. **Web Search:** If the query requires information not in the memory, it sends a request to the web search. 5. **AI Model Selection:** Based on the query's complexity, the system chooses either AI Model 1 or AI Model 2. 6. **Response Generation:** The selected AI model generates a response. 7. **TTS:** The text response is converted to speech. 8. **Output:** The assistant speaks the response. 9. **Memory Update:** The interaction and response are stored in memory. **Model Selection:** * **Simple Requests:** AI Model 1 (like me) handles simple questions, greetings, and common tasks. * **Complex Requests:** If the query involves complex reasoning, specialized knowledge, or creative tasks, AI Model 2 is used. * **Hybrid Approach:** Sometimes, both models could work together - Model 1 for initial processing and Model 2 for refining or expanding on the answer. It’s quite a bit of fancy technology to make an assistant, isn't it?