Talking to Pepper by phone
Once the Android bridge, a spare Android phone plugged into the Mac, is set up, Pepper answers that phone's number.
What happens depends on the number calling. Pepper checks it against data/phone/callers.json on the Mac that
runs it (how to create it), comparing the last ten digits.
| Caller | What happens |
|---|---|
| You, from your own number | Pepper asks for your spoken passphrase. Once you say it, you can ask questions and add tasks. |
| A contact you listed | Pepper says it's your AI assistant and that the call is transcribed, takes a message, and sends it to you on Telegram. |
| Anyone else | Voicemail: the same introduction, a tone, one message, sent to you on Telegram. |
A caller can't learn anything from Pepper, however they ask: calls from contacts and strangers never reach Ask or anything Pepper knows. Their audio does go to the transcription model, on the Mac unless you chose OpenAI, along with the first names of people Pepper has seen, so a message is transcribed more accurately.
Caller ID can be faked, so your number alone unlocks nothing. Pepper also asks for a short phrase only you know, every time you call.
You set it when you turn calls on. The last setup step, scripts/phone/pepper-phone.sh on, asks for it
when your number is in callers.json and no passphrase is set yet, so Pepper can't start answering your calls
without one. You type it twice, with the text hidden.
To set it separately, open a terminal on the Mac that runs Pepper and run this from the Pepper folder:
npm run phone -- passphraseIf you're at a different Mac from the one that runs Pepper, connect to it over SSH and run the command there.
The -t lets the hidden prompt work:
ssh -t you@your-mac 'source ~/.nvm/nvm.sh && nvm use 24 && cd ~/pepper && npm run phone -- passphrase'Pick three or more ordinary words you can say clearly, for example a short phrase only you would use. Pepper compares words, not spelling:
- Capitals and punctuation don't matter.
- Digits count as words: "4" and "four" are the same.
- A few extra words before it are fine ("okay, it's …"), as long as you say your words together, with nothing in between.
Check it with npm run phone -- callers. It shows whether the passphrase is set. npm run doctor and
scripts/phone/pepper-phone.sh status also flag a missing passphrase while calls are on.
Change it by running npm run phone -- passphrase again. The new one replaces the old one straight away.
Remove it by deleting data/phone/passphrase. Your calls then get the "isn't set up" message below until you set
a new one.
How it's stored. Pepper stores only a salted scrypt hash, a one-way scramble it can check your phrase
against, in data/phone/passphrase, readable only by your user account on the Mac. The file is never part of the repo, and
the phrase is never logged. When you say it on a call, the recording and its transcript exist on the Mac only as
long as the check takes, then are deleted.
Who else can hear it. Between the bridge phone and the Mac, call audio crosses your home Wi-Fi unencrypted. Someone who records a call could replay the phrase while faking your number. Treat it like a door code, and watch for the Telegram message Pepper sends as soon as someone says the right phrase.
Repeated failures lock it. Each call gets three tries. If two calls from your number use up all three within an hour, Pepper stops asking for an hour and tells you on Telegram. If those calls weren't you, someone may be faking your number. A call where nothing was said doesn't count. To clear the lock sooner, on the Mac:
npm run phone -- unlockSay your passphrase only when Pepper asks for it on a call you placed. Pepper never asks for it on Telegram, in a message, or on a call it makes to you.
When you call
| Moment | Pepper says |
|---|---|
| No passphrase set yet | "Hi, it's Pepper. Your phone passphrase isn't set up yet, so I can't talk about anything private. Set it up on the Mac, then call me back. Bye." You also get a Telegram message with the command. |
| Answering | "Hi, it's Pepper. What's the passphrase?", then a short beep. Say it after the beep: anything said before it isn't heard. |
| Wrong phrase | "That's not it. Try once more." (or "I didn't catch that" if it heard nothing), then the beep again |
| Third wrong try | "Sorry, I can't continue this call. Bye." You get a Telegram warning that a call from your number failed the passphrase check, in case it wasn't you. |
| Passphrase locked | "Sorry, I can't continue this call. Bye." at once, and a Telegram message saying npm run phone -- unlock clears it. |
| Right phrase | "Thanks. What can I do?" You also get a Telegram message at once: 📞 Someone passed your phone passphrase at 5:12 PM and is talking to me now. If that isn't you, send /halt. |
Pepper is paused (/halt) | "Pepper is paused right now. Try again later. Bye." This is the same for every caller, and no message is taken. |
Pepper waits up to six seconds for you to start talking, and treats a pause of about a second and a half as the end of your passphrase. Say it in one go, without stopping between words.
Then talk normally. Each thing you say can run up to 30 seconds; a pause of just over a second ends your turn, and if you say nothing for seven seconds Pepper asks "Still there?".
- Ask anything you'd ask on Telegram: "What do I owe Sam?", "Anything from Dana today?". Pepper aims for a few short spoken sentences. If an answer takes more than a couple of seconds, Pepper says "One moment."
- Add tasks by saying them: "Remind me to call the dentist tomorrow." Pepper repeats what it saved ("Got it: call the dentist.") and asks "Anything else?". After the call, Telegram lists how many tasks were added.
- End the call by hanging up or saying "bye", "that's all" or "I'm done".
- If you stay quiet after "Still there?", Pepper says "I'll let you go. Bye."
- After the passphrase, Pepper ends the call between exchanges once 15 minutes have passed or after 40 exchanges.
- Sending
/halton Telegram during a call ends it at the next exchange: "Pepper is paused now. Bye."
When a contact calls
"Hi Sam, this is Pepper, [your name]'s AI assistant. This call is transcribed so I can pass it on. [Your name]
can't come to the phone right now. What should I tell [your name]?" Then it asks "Anything else?" once, thanks
them and hangs up. You get the message on Telegram: 📞 A call from Sam's number (caller ID can be faked) at 4:12 PM: "…". Pepper only knows the number, so the message doesn't claim it was Sam.
When anyone else calls
"Hi, you've reached Pepper, [your name]'s AI assistant. This call is transcribed so I can pass it on. Please say your name and your message after the tone." Callers can leave one message of up to 90 seconds, which Pepper sends to you on Telegram as a voicemail.
- Audio: each thing a caller says is recorded only long enough to turn it into text, then deleted. If
Pepper stops in the middle of a call, any leftover recording is deleted when it next starts; a crash can leave
a copy of the clip in the Mac's temporary folder until macOS clears it. Speech is turned into text on the Mac,
unless you set
PEPPER_STTto use OpenAI, in which case messages and your questions go to OpenAI. - Spoken replies: fixed lines, like the greeting and "Still there?", are kept as audio in
data/phone/cacheso they play at once. Lines unused for 30 days are removed when Pepper starts. Answers to your questions and anything else private are made for the call and deleted once played. - Messages and voicemail: the transcribed text is sent to you on Telegram.
- Your own calls: what you say is handled like a Telegram message. Questions go to the model and tasks go on your list. Both are kept the way Telegram messages are: the conversation in Pepper's database on the Mac, and each task with the words it came from.
- The log (
data/pepper.log) shows only the last two digits of a caller's number, who they were (owner, contact, unknown) and how many exchanges there were. Never what was said.
These all go in Pepper's .env settings file on the Mac. Restart Pepper after changing them.
| Setting | Default | What it does |
|---|---|---|
PEPPER_PHONE | off | on makes Pepper listen for calls from the call server. The log then shows [phone] waiting for calls on 127.0.0.1:8084. The phone hands its calls over only after scripts/phone/pepper-phone.sh on; both are needed |
PEPPER_PHONE_PORT | 8084 | The local port where FreeSWITCH, the call server on the Mac, hands calls to Pepper. Only change it together with the FreeSWITCH config |
PEPPER_PHONE_SILENCE | 200 | How loud speech must be to count as speech. Lower it if Pepper cuts you off; raise it if line noise keeps it listening |
| Symptom | Cause | Fix |
|---|---|---|
command not found: npm when setting it over SSH | SSH sessions don't load nvm | Use the SSH command above, which loads it first |
| "That's not it" even though you said it | Speech-to-text heard a different word, or a word is missing | Speak a little slower. If one word is often misheard, choose a new phrase with plainer words |
| "I didn't catch that" while you're talking | The line is quiet, so your speech is below the threshold | Lower PEPPER_PHONE_SILENCE, for example to 120 |
Every call fails at once, and the log says refused a connection without the dialplan token | The pepper_token in FreeSWITCH's dialplan, its call-handling rules, doesn't match data/phone/token | scripts/phone/pepper-phone.sh mac --config-only, then status |
| Pepper treats your number as unknown | The number in callers.json doesn't match what the carrier sends | npm run phone -- callers shows the last four digits. Write numbers with the country code, like +14155550123 |
| Callers hear "Sorry, Pepper can't take calls right now. Please try again later." | Pepper isn't running, or PEPPER_PHONE isn't on | Check the log for [phone] waiting for calls. Restart Pepper |
| The bridge phone just rings on the desk | Calls aren't turned on (takeover is off), so the phone acts like a normal phone | scripts/phone/pepper-phone.sh status, then on |
Phone line: Android bridge (recommended)
A spare Android phone with your SIM card sits next to the Mac, plugged in by USB. When someone calls that number, an app on the phone, the bridge, hands the call to the Mac. Pepper hears the call and…
Setting up the Mac that runs Pepper
Pepper runs on one always-on Mac, usually a Mac mini. Install it with the install guide; this page covers the settings on the Mac, so Pepper keeps running through idle nights, power cuts and…