{"id":151,"date":"2015-09-27T19:55:49","date_gmt":"2015-09-27T19:55:49","guid":{"rendered":"https:\/\/carson.fenimorefamily.com\/?p=151"},"modified":"2015-09-28T03:08:22","modified_gmt":"2015-09-28T03:08:22","slug":"using-raspberry-pi-for-two-way-videoaudio-streaming","status":"publish","type":"post","link":"https:\/\/carson.fenimorefamily.com\/?p=151","title":{"rendered":"Using raspberry pi for two-way video\/audio streaming"},"content":{"rendered":"<p>I am currently writing custom software to create a variety of distributed media solutions. \u00a0One of these involves a raspberry-pi security camera. \u00a0Right now my software isn&#8217;t finished, but I wanted to prove the concept out using existing software. \u00a0System requirements:<\/p>\n<p>Raspberry pi:<\/p>\n<ul>\n<li>Broadcast audio (over the LAN) to a server<\/li>\n<li>Broadcast video (over the LAN) to a server<\/li>\n<li>Receive (and play over the speaker) audio<\/li>\n<\/ul>\n<p>Laptop:<\/p>\n<ul>\n<li>Receive audio\/video from the raspberry pi (and display it)<\/li>\n<li>Send audio (from the laptop&#8217;s own microphone) to the pi.<\/li>\n<\/ul>\n<p>I bought a raspberry pi b+, cheapo USB microphone, and IR camera (with built-in LEDs), and some speakers. \u00a0The whole package was around $100, which is just plain awesome. \u00a0Im sure my grandkids will marvel, 40 years down the road, that for the cost of an Obama-approved school lunch, I could buy so much.<\/p>\n<p>Here&#8217;s a schematic of the setup:<\/p>\n<p><a href=\"\/\/carson.fenimorefamily.com\/wp-content\/uploads\/2015\/09\/design.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-153\" src=\"\/\/carson.fenimorefamily.com\/wp-content\/uploads\/2015\/09\/design.jpg\" alt=\"design\" width=\"1221\" height=\"1104\" srcset=\"https:\/\/carson.fenimorefamily.com\/wp-content\/uploads\/2015\/09\/design.jpg 1221w, https:\/\/carson.fenimorefamily.com\/wp-content\/uploads\/2015\/09\/design-300x271.jpg 300w, https:\/\/carson.fenimorefamily.com\/wp-content\/uploads\/2015\/09\/design-1024x926.jpg 1024w\" sizes=\"auto, (max-width: 1221px) 100vw, 1221px\" \/><\/a><\/p>\n<p>This can be accomplished with a few scripts, which i based heavily on those found here: http:\/\/blog.tkjelectronics.dk\/2013\/06\/how-to-stream-video-and-audio-from-a-raspberry-pi-with-no-latency\/<\/p>\n<p>Run the following on the pi:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nfunction on_signal (){\r\n    launch &quot;kill -9 $piVideoPid $piAudioPid $osxAudioPid&quot;\r\n}\r\ntrap 'on_signal' EXIT\r\n\r\n# Receive os x audio\r\ngst-launch-1.0 -v udpsrc port=9102 caps=\\&quot;application\/x-rtp\\&quot; ! queue ! rtppcmudepay ! mulawdec ! audioconvert ! alsasink device=plughw:ALSA sync=false &amp;amp;\r\nosxAudioPid=$!\r\n\r\n# Send pi audio\r\ngst-launch-1.0 -v alsasrc device=plughw:Device  ! mulawenc ! rtppcmupay ! udpsink host=$broadcastIp port=$piAudioPort &amp;amp;\r\npiAudioPid=$!\r\n\r\n# Send pi video\r\nraspivid -t 999999 -w 1080 -h 720 -fps 25 -hf -b 2000000 -o - |  gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96  ! gdppay ! tcpserversink host=$localIp port=$piVideoPort\r\n<\/pre>\n<p>Run the following on the\u00a0laptop (a macbook in my case). \u00a0Note if this were a linux box you would probably use &#8220;autoaudiosrc&#8221;.<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nfunction on_signal (){\r\n    launch &quot;kill -9 $piVideoPid $piAudioPid $osxStreamPid&quot;\r\n}\r\ntrap 'on_signal' EXIT\r\n\r\npiIp=&amp;lt;raspberryPiIp&amp;gt;\r\n\r\n# Stream OSX microphone (note the capsfilter, apparently needed due to os x bug)\r\nlaunch &quot;gst-launch-1.0 -v osxaudiosrc ! capsfilter caps=audio\/x-raw,rate=44100 ! audioconvert ! audioresample ! mulawenc ! rtppcmupay ! udpsink host=$broadcastIp port=$macAudioPort&quot; &quot;ASYNC&quot;\r\nosxStreamPid=$!\r\n\r\n# Receive audio\r\nlaunch &quot;gst-launch-1.0 -v udpsrc port=$piAudioPort caps=&quot;application\/x-rtp&quot; ! queue ! rtppcmudepay ! mulawdec ! audioconvert ! osxaudiosink&quot; &quot;ASYNC&quot;\r\npiAudioPid=$!\r\n\r\n# Receive video\r\nlaunch &quot;gst-launch-1.0 -v tcpclientsrc host=$piIp  port=$piVideoPort ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false&quot; &quot;ASYNC&quot;\r\npiVideoPid=$!\r\n\r\necho &quot;Waiting on $piVideoPid $piAudioPid $osxStreamPid&quot;\r\nwait\r\n\r\n<\/pre>\n<p>You should be able to see and talk to whoever is on the other end of the pi. Biggest problem is audio feedback, if the laptop and pi are close to one another. \u00a0I havent even begin to think about this, but the coolest things to note are:<\/p>\n<ul>\n<li>You get a silky-smooth h264 video<\/li>\n<li>The audio is acceptable<\/li>\n<li>It works almost entirely using gstreamer. \u00a0The cool thing about that is gtreamer has a very easy programmatic API. \u00a0So&#8230; that makes my job easy when I get to that part of my project.<\/li>\n<\/ul>\n<p>Enjoy!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I am currently writing custom software to create a variety of distributed media solutions. \u00a0One of these involves a raspberry-pi security camera. \u00a0Right now my software isn&#8217;t finished, but I wanted to prove the concept out using existing software. \u00a0System requirements: Raspberry pi: Broadcast audio (over the LAN) to a server Broadcast video (over the &hellip; <a href=\"https:\/\/carson.fenimorefamily.com\/?p=151\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Using raspberry pi for two-way video\/audio streaming<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-151","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/posts\/151","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=151"}],"version-history":[{"count":3,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/posts\/151\/revisions"}],"predecessor-version":[{"id":169,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=\/wp\/v2\/posts\/151\/revisions\/169"}],"wp:attachment":[{"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=151"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=151"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/carson.fenimorefamily.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=151"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}