Raspberry Pi - A credit card sized, linux based mini-computer for $30

I'd never looked into how the kinect works. It seems that it has an illuminating IR LED and then an ASIC to quickly process what the camera is seeing reflected back using a trained machine learning algorithm. So it is like the deluxe version of my "stupid" idea for the ultrasound sensors.

those are all really good ideas! it looks like i'll have some options. tbh at the moment i am planning on it tracking only one (the closest) person. i'm working on some video cycling logic that lets it smoothly track from one target to another, so if one person is the target and then they suddenly turn around and leave, the eyes/face will smoothly work its way over to the next-closest person detected (or to it's default resting cycle if no one else is there).

i won't break any ground technically but i hope to make the videos interesting and transition from one another smoothly

have any of you used the Pine64 (https://www.pine64.org/)? or heard anything about it from someone who has?

i'm working on something involving speech recognition and synthesis and while i have the guts of it working on my raspberry pi 3, it's painfully slow. a 2 second speech recording takes about 6 seconds to convert to text, and that's before i even do text-to-speech. meanwhile, i can run the same python code on my mac and the conversion is near-instantaneous. that suggests to me that the slowdown is due to the lack of specs on the raspberry pi. fishing around for a way to improve the performance, i saw some chatter about the relatively new pine64, which apparently has more processing power and general capabilities while still in the $30-50 range.

or actually...maybe it would make more sense to look into inexpensive ways to augment/power up the RPi somehow?

How do you want to install the final project? Is it feasible to use the Pi as a client and offload the speech-to-text to a faster computer over the network? I can well imagine it being hard to get low-latency speech to text performance on any cheapo device.

really hoping to make it as modular and mobile as possible, like completely independent and offline units that could be arranged in different ways. i wanted offline because it has to run for a few hours straight and i didn't want the whole thing to break if the wi-fi went down or something.

Is it feasible to use the Pi as a client and offload the speech-to-text to a faster computer over the network?

maybe? i think it would work, but i don't know how much faster it would be. each Pi unit has its own USB microphone to pick up the last signal in the chain, and each unit also has it's own speaker output connected to it. So each pi unit would have to send the recorded audio file to the faster computer, which would convert it (very quickly), but then send the converted file back to the pi unit again before playing it through the speaker. it just seems like the wait times for sending it back and forth would negate the improvement in speech-to-text conversion speed.

Sensible of you. I think you might want to treat this as a “prototyping” stage and live with what $50 gets you. It looks like for $500 you can get a development board for the NVidia Tegra X1, which is the chip the Nintendo Switch uses and probably has several orders of magnitude better performance than a Pi. Expensive but still cheaper and smaller than duct-taping a MacBook to something.

Don’t overestimate the round trip time of the network though. Six seconds is a long time.

oh, i'll definitely give it a shot! i'm also going to look into adjusting the dictionary i'm using for the speech recognition (using Sphinx). i saw a video of some dude online who is getting a decent turnaround on his RPi3, so maybe he was using a limited dictionary?

this stuff is sometimes frustrating but really fun to look into (especially when it works in the end)

Are there any other speech recognition softwares you've tried?
If the idea is to just robotize people's speech, maybe have it recognize/generate phonemes instead of words?

all the other speech recognition i've found requires an API: a couple google versions limited to 50 API calls a day, IBM, Bing, Jasper, Houndify, others.

i've been thinking of a way to take advantage of the slow processing. i'm thinking a set of 4 units that each work on 12 second cycles: 2 for listening to speech, 6 for converting to text, 2 for text-to-speech playback, and some buffer time in between each of those steps. the timing of the units would overlap so there'd be playback every 2 seconds. like this:

https://i.imgur.com/6afC7mA.png

(sorry for ugly colors)

actually, the cycle length would vary, but each would loop every 32 seconds with the timeline above. i'm thinking i can use python's time.time() to keep everything on schedule, so even if individual components in the cycle (record, convert, playback) take longer or shorter than expected, each step should begin at a precise point in time. the biggest issue (i think) will be with calibrating the units at the beginning. i'm thinking i can start the first unit on it's eternal loop, then manually trigger the launch of the second unit, listen for a few cycles to make sure they're in alignment, continue to the 3rd, and then the 4th.

if all that works i hope to include to introduce human speech (whoever is in the room with this thing) into the mix as well

it's eternal loop

jfc, its

is the idea that these units get a lot of crosstalk from each other so it ends up as cacophonous siri feedback, or are they more orchestrated?
would it be bad if they get slightly out of phase?

the goal was to try to make it tightly coordinated, but i was i interested in what would happen when they inevitably went out of phase.

but shit, bad news. i thought the easy part would be the speech synthesis, because as I already had that working on my mac. turns out that audio and raspberry pi don't work well together at all - playback is wildly irregular, almost always cutting off pieces of sentences, and sometimes starting playback a second into the .wav file, or cutting off the last bit. it sounds the same whether i use aplay or omxplayer to play sound, or the native functions of TTS software like festival, festival lite, espeak.

i don't think it's me, either, i think it's the RPi. reading through this thread (https://www.raspberrypi.org/forums/viewtopic.php?f=38&t=47942) the guy who nails it is the pair of posts from nPHYN1T3 on page 2. i'm feeling you nPHYN1T3, i'm right there too

sorry for drunk. but i'm kind of super bummed because i thought the RPi would be powerful enough to at least play sound files on command. if it can't do that i probably need to start over.

Seems like these weenies will sell you an outboard DAC board for your Pi?

https://www.hifiberry.com/products/dacplus/

Tho to be clear I have no idea if that would help at all lol

hmm...it might! i was thinking about that earlier (actually, i was thinking of connecting to an arduino and using a wave shield, which would just be one more unnecessary step). i'm not sure if it would work either. it's not so much the sound quality, it's more just the weird thing about the beginning and ending of audio clips getting cut off. super weird. it would be one thing if the cutoff was predictable, because then you could just build in extra silence at the edges. but sometimes it cuts off half of the whole file, sometimes just the first second, sometimes only the first syllable. weird.
i might give the DAC a shot. thanks!

Isn’t there supposed to be like some media decoding in hardware on these chips? This is perverse but does an mp3 play back without an issue?

are you playing back audio through the HDMI or headphone jack? also, it might be a stretch but someone had figured out how to use raspberry pi to transmit FM radio (attach a bit of wire to one of the pins to act as an antenna) so that might be another way of getting generated audio off the pi.

Same problem with wav and mp3, yeah.

Same problem with 3.5 mm output. It’s really the craziest thing, seems like people have been complaining for years too.

I will troubleshoot some more tomorrow, but for tonight I declare temporary failure

Temporary failure is just the prelude to permanent success!

—Steve Jobs

I could track it down, but I had a cheap dac that was pretty darn good connected to a pi for quite a while. As in, purchased on alibaba and available in bulk cheap. I don’t remember any playback issues but I was playing music and not short clips

is it only happening on starting and finishing an audio clip? maybe keep the audio engaged by playing an infinite silent stream simultaneously so there's never a start/stop?

Philip i did run across a proposed solution along those lines, and i'm going to give that a shot. plan B (actually more like Plan Z-42 at this point) is going with an audio DAC. i'm also going to try to see if there are any maker-oriented physical stores in the Chicago area.

i also want to look into boosting the processing power - i'm kind of wary to overclock because my normally-clocked unit is having issues with power (if i have a USB microphone connected and then run chromium at the same time, the pi crashes). but there might also be ways to use all 4 cores of the pi rather than just one (this thread has several ideas along these lines: https://www.raspberrypi.org/forums/viewtopic.php?t=198425)

i was super bummed about this the other night but now i'm back in the saddle.

thanks for the ideas and help, by the way! really appreciate it.

are you playing back audio through the HDMI or headphone jack?

i tried the headphone jack again, this time by just right-clicking the desktop GUI sound icon and changing it to Analog. It worked! At the moment it's not cutting off any text, and isn't taking too long to process the TTS either.

i'm going back to the voice recognition (via eSpeak) part again tonight, only this time my plan is to use a pretty limited dictionary/language model (around 500 words) rather than trying to attempt to recognize everything in English. i think that might have been the source of the slow processing speed (it was taking about 6 seconds to process 2 seconds of audio).

Is it feasible to use the Pi as a client and offload the speech-to-text to a faster computer over the network? I can well imagine it being hard to get low-latency speech to text performance on any cheapo device.

― direct to consumer online mattress brand (silby),

silby, i'm a fool. i think this could work! i wanted to avoid dependence on the internet because i didn't want some sort of uncontrollable network connection issue to break it. but a friend just explained to me that it's possible to my computer as the server, the raspberry pi's as clients, and a local modem to transmit the wi-fi signal, without my computer needing to be connected to the internet itself. right? i'm a noob in pretty much everything i'm doing here so i might be way off, but it seems like the only possible connection issue would be the building's wi-fi interfering with the local wi-fi between my laptop and the RPIs, and even that would be avoidable by just using a unique port (or something) on my router?

with all that said, i'm viewing all of that as a backup option, and i'd still rather figure out a way to get these devices working together without any network, as fully independent links in the chain. but if i'm not able to do it, it'll be great to have the server/client processing as an option.

a minor victory today! i didn't realize that time.time() is so consistent and precise between UNIX devices. when i call it on either my laptop or pi, they return the same exact value (in seconds since jan 1, 1970..siiiick). i can run the following bit of code on two different devices, and they'll both play the sound at the exact same time, independently.

import time
import subprocess
def Launch():
	print ("unit initialization time: ", time.time())
	print("waiting for next 18-second cycle to begin...")
	while time.time() % 18 > abs(.005):
		time.sleep(.001)
def Cycle():
	while True:
		print("new cycle: ", time.time())
		subprocess.call('afplay test.wav', shell=True)
		while time.time() % 18 > abs(.005):
			time.sleep(.001)
Launch()
Cycle()

the way i'm making it wait for 18 second intervals is a little hacky but it works.

what's nice is that i can start up one device and then wait as long as i want to start the second, and they'll both sync up as soon as the start of the next 18-second cycle comes around. to me, this is exciting because it means i can define various events in an endless 18-second loop (play a message, record a message, choose a new phrase and play it back, etc), and even if the processing speeds for the various events vary from unit to unit, i can still specify exact times in the cycle for the events to happen so that they stay in sync. plus, it means that the entire chain of devices can be modular, constrained only by the cost of buying all the equipment and the size of the room.

while time.time() % 18 > abs(.005):

why abs() on a constant positive value?

positive or negative, it basically sets a range of .010, centered on 0, as a kind of buffer. i tried setting it at exactly 0 but realized that the granularity of time.sleep is only around 1ms or so, whereas time is measured down to 7 decimal places (the current time is 1520489816.0669749, in seconds since january 1, 1970), which means that it's very unlikely for the time to every hit exactly 0.0000000. so i'm basically establishing a trigger zone for the time rather than requiring an exact value. it seems to work - the output is showing me that i'm triggering new events within thousandth of a second of the "real" time, and it works with multiple, independent units in perfect sync. pretty trippy tbh!

actually i guess time's not going to be negative, so you're right koogs, just positive! but either way the principle is kinda the same, it's just establishing a buffer/trigger zone.

you don't have a buffer there, the absolute value of .005 will always be .005

if you have a variable in there and it could be either .005 or - .005 it'd always return the positive value

absolute value is math shorthand for "drop the negative sign if there is one"

Ok, including absolute value was a silly goof (it can just be > .005) but it still manages to do the job of waiting to start the next cycle until the remainder is between 0 and 0.005, which is the “buffer” I’m blabbing about. Part of my problem in life is I just loosely use terminology that actually has specific meaning in the discipline I’m flailing in atm

You might be able to reduce the number of Pis in favor of cheap radios using something like this:
https://www.youtube.com/watch?v=NdjQw6P0HLA

Also, does the behavior change if you do something like this?:

print ("unit initialization time: ", time.time())
print("waiting for next 18-second cycle to begin...")
while True:
  time.sleep(18 - (time.time() % 18))
  print("new cycle: ", time.time())
  subprocess.call('afplaytestest.wav', shell=True)

should be .wav',shell=True not wavhell=True unless it's some kind of black metal track

Also, does the behavior change if you do something like this?

it does work! and it does simplify the code a bit, too. t's funny though, because it slightly decreases the precision of the cycle starts. here's the log for time.sleep(5 - (time.time() % 5)):

launch time:  1520541567.056979
waiting for next 5 -second cycle to begin...
new cycle:  1520541575.004256
new cycle:  1520541580.005019
new cycle:  1520541584.999948
new cycle:  1520541590.005019
new cycle:  1520541595.004566
new cycle:  1520541600.0001438
new cycle:  1520541605.000339
new cycle:  1520541610.0040052

which is great! accurate to the ms unit, for the most part. but then here's the log for the while time.time() % 5 > 0.005 approach:

launch time:  1520541896.142796
waiting for next 5-second cycle to begin...
new cycle:  1520541900.000105
new cycle:  1520541905.000193
new cycle:  1520541910.001322
new cycle:  1520541915.000451
new cycle:  1520541920.000237
new cycle:  1520541925.000406
new cycle:  1520541930.0009248
new cycle:  1520541935.000673

accurate to another degree, for some reason. anyway, it doesn't really matter since all of that is way more precise than i actually need it to be.

the fm transmitter idea is really cool! i'm not sure i can really use it for this because i want each unit to have both a speaker and a microphone. each one needs to be capable of recording audio using a microphone, translating to text, converting that back to speech (TTS), then playing it over the speaker. i've already got a working version with all of that, it just needs to be tweaked quite a bit (mostly with the speech recognition side - optimizing dictionary and language models in particular) . i've been so busy the last week or two but i need to just go ahead and order the other units + cheap but good USB microphones + cheap speakers so that i can do the tweaking with the real units.
i've been so busy the last week with various things so i haven't been

i meant to say though, i can think of all sorts of things you could do with the FM transmitter setup, though, as a separate project. i feel like there's all sorts of untapped potential in these things.

of course, the day after i buy 4 new raspberry pis, they announce a newer, better, faster version, at the same price:

https://www.raspberrypi.org/blog/raspberry-pi-3-model-bplus-sale-now-35/

i am thinking of getting one of these for me and my son as a project, initially for video game emulation. anyone use one for that purpose?

i haven't personally, but i think several people in this thread have before! and plus, out of all raspberry pi project ideas, the emulation one is probably the most popular and has many online guides you can use to walk you through the process. i think it's a fantastic project idea for you and your son, because it's complicated enough to be interesting and show you some of the possibilities of the device, but also you have like a 99.5% chance of succeeding within a reasonable time frame.

I have one hooked up to my TV that I use as a video game emulator (classic arcade + consoles). Works really well, running RetroPie. Connected to the TV via HDMI and the built-in bluetooth pairs with a wireless Nintendo Pro controller.

xp to KM yeah! ive been looking at some of the youtube tutorials for it. i guess im sold on getting one, but not sure if want to spend a little extra to get the 4GB ram on the pi 4 or be low budget and get a 3B+. i guess i dont want to mess with several hundred gigs of games and will probably stick to older consoles (maybe up to N64/PS1?)

Pro Switch controller ?

Xp