API for live streaming about arlo HOT 63 CLOSED

So, got a chance to look into this issue a little deeper. The streaming protocol they use is rtmps (Real-time Messaging Protocol). I guess I never really tested this function...

Streaming rtmps is kind of complicated and thus, will require a 3rd-party library (https://pypi.python.org/pypi/python-librtmp). All of the libs I have found so far rely on librtmp under the hood, which I have had trouble building on some of my machines.

In order to keep things simple, I'm thinking I might just return the rtmps info you'll need to get the stream. And let the calling code use whatever library they want to actually do the streaming.

If there's strong demand for this library to implement the rtmp streaming, please vote for it and I'll make it happen.

from arlo.

It looks like the format has changed slightly - you now need to specify properties as: "properties":{"activityState":"startUserStream","cameraId":deviceId} and also pass the cloud id in the headers like with the Arm/Disarm functions. This gets you the rtmps URL back. Although, I haven't yet been able to do anything useful with it.

from arlo.

Thanks, Jeffrey!
This helps a lot.

from arlo.

Sorry for the delayed response. Been super busy moving. I will try to test things out and respond tomorrow.

from arlo.

@jeffreydwalter Or course! I am planning to use Arlo on an interstate highway to detect trucks for my research project. I am trying to use yolo and SIFT to process images. I can definitely share the code with you once done.

from arlo.

Hi garethx,

Thanks for your response!

May I ask where I could get the cloud_id?
Thanks!

from arlo.

I adapted the basestation filter used in the Arm/Disarm example code to get the cloud id for the Arlo Q camera: arloq = [ device for device in arlo.GetDevices() if device['deviceType'] == 'arloq' ]

You can then get the cloud id you need and pass it to stream, like so stream = arlo.StartStream(body, arloq[0]['xCloudId'])

A working body example is: body = { "to":arloq[0]['deviceId'], "resource":"cameras/"+arloq[0]['deviceId'], "action":"set", "publishResponse":"true", "properties":{"activityState":"startUserStream","cameraId":arloq[0]['deviceId']} }

Which I pass to StartStream, and that looks like:
def StartStream(self, body, xcloud_id):

	body['transId'] = self.genTransId()

	body['from'] = self.user_id+'_web' 

	body = self.post('https://arlo.netgear.com/hmsweb/users/devices/startStream', body, 'StartStream', headers={"xCloudId":xcloud_id})

from arlo.

Thanks a lot!

from arlo.

I removed the StartStreaming() method and added a GetStreamUrl() method in it's place. See this commit: 26d80d5

I personally feel like this is a better solution because it allows you to use whatever streaming library you want.

from arlo.

@rayhou0710 I was interested in doing some OpenCV object detection work on the Arlo videos, mainly to reduce the number of "false" alerts I get from Arlo. Would you be interested in collaborating or sharing the work you're doing?

from arlo.

@jeffreydwalter By the way, may I ask how you reversed engineered the Arlo system? (Kind of silly question...) I took some classes on web and database before, but I am always under the impression that you can only get/post things using given urls. I don't know how you got all the fields, links, etc... I am super curious. I would appreciate if you could give some hints.

from arlo.

Not a silly question at all. :) I used the inspector panel in Chrome to view all of the http requests that were being made by the browser when I clicked through all of the buttons on the arlo.netgear.com website. The browser inspector panel allows you to see all of the headers that are sent/received, along with the request/response body data, and urls.

It is a tedious process, but not really all that difficult.

from arlo.

Thanks for all the info on getting the streaming Url. I have successfully been able to interface this with my cameras.

But I'm stumped on how to decode the RTMPS stream. As it's encrypted with an SSL certificate, do I need access to the certificate to decrypt? I haven't been able to decode the stream so far in any app :(

from arlo.

So I was poking around in the Arlo APK for android, and found a wowza netgear.com certificate. Wondering if this is the key?

from arlo.

from arlo.

Poked around a little bit at the tag being created for the .swf player. Looks like they embed the "token" referenced in the flowplayer docs I pasted previously. It's in the flashvars parameter of the object element.

<object width="100%" height="100%" id="XXXXXXXXXXXXX_api" name="XXXXXXXXXXXXX_api" data="flowplayer/flowplayer.commercial-3.2.18.swf" type="application/x-shockwave-flash">
<param name="allowfullscreen" value="true">
<param name="allowscriptaccess" value="always">
<param name="quality" value="high">
<param name="bgcolor" value="#000000">
<param name="wmode" value="opaque">
<param name="flashvars" value="config={"key":"#$XXXXXXXXXXXXXXX","debug":false,"log":{"level":"error","filter":"*"},"play":{"opacity":0},"clip":{"autoPlay":true,"autoBuffering":true,"url":"XXXXXXXXXXXX_XXXXXXXXXXX","live":true,"provider":"influxis","connectionProvider":"secure","bufferLength":1},"plugins":{"influxis":{"url":"flowplayer/flowplayer.rtmp-3.2.13.swf","proxyType":"best","netConnectionUrl":"rtmps://vzwow124-z2-prod.vz.netgear.com:80/vzmodulelive?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXX&userAgent=web&cameraId=XXXXXXXXXXXX_XXXXXXXXXXX"},"secure":{"url":"flowplayer/flowplayer.securestreaming-3.2.9.swf","token":"THISISALONGHEXSTRINGWHICHISTHETOKENYOUPROBABLYNEED"},"controls":null},"playerId":"XXXXXXXXXXXX","playlist":[{"autoPlay":true,"autoBuffering":true,"url":"XXXXXXXXXXXX_XXXXXXXXXXX","live":true,"provider":"influxis","connectionProvider":"secure","bufferLength":1}]}"></object>

from arlo.

Yeah, I found this token as well.

I'm trying to work out how to replicate the connection that flow player is using to the wowza service. I'll try checking out the wowza site and see if they have any docs.

from arlo.

Checkout this link https://www.wowza.com/docs/how-to-protect-streaming-using-securetoken-in-wowza-streaming-engine

from arlo.

And this one http://flash.flowplayer.org/demos/plugins/streaming/secure-wowza.html

from arlo.

Luckily for us, it seems Netgear didn't really put much effort into securing their token. :)

So, we have the url and token:

"netConnectionUrl":"rtmps://vzwow124-z2-prod.vz.netgear.com:80/vzmodulelive?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXX&userAgent=web&cameraId=XXXXXXXXXXXX_XXXXXXXXXXX"
"token":"THISISALONGHEXSTRINGWHICHISTHETOKENYOUPROBABLYNEED"

We just need to work out how to get the token from Python and what to do with these two to get the actual stream.

from arlo.

@ryanwinter did you have any luck?

from arlo.

hey @jeffreydwalter, I looked at this a bit but couldnt get anything to work. I used the ios user agent which generated a rtsp type link on port 443. I tried using ffmpeg and the rtsps protocol to connect but received a 404 error.

A 404 seemed a positive step in that at least its generating a valid response, not sure if that was real or just a catchall. Was planning to add some more debugging to ffmpeg to see exactly whats going on.

If the 404 is real, is could be that I am not passing the username/password in with the url or something. I plan to do a little more investigation but it's pretty slow going :D

from arlo.

@ryanwinter how did you get the rtsp link? I tried changing '_web' in this call but no luck. 404 error could also be just dead url. 403 maybe if username/pass is problem. But I think going after iOS is the right approach. The flowplayer RTMPS extra handshake/encryption with built-in encryption is a problem.

self.post('devices/startStream', {"to":device_id,"from":self.user_id+"_ios","resource":"cameras/"+device_id,"action":"set","publishResponse":True,"transId":self.genTransId(),"properties":{"activityState":"startUserStream","cameraId":device_id}}, 'StartStream', headers={"xcloudId":xcloud_id})

from arlo.

Any update on the significance of flashvars to make librtmp load the secure player plugin?

from arlo.

@ryanwinter @jeffreydwalter Curious if either of you are working on this at all anymore. I have been playing around with getting the live stream to no avail as well. I think the key may be the link @jeffreydwalter posted above regarding the secureToken. I am thinking that the token we get in flashvars may be the shared secret from which we need to get the hash using some set of parameters to get the actual url for the stream. Unfortunatly I haven't found a way to sniff the actual rtmps stream so I am uncertain what url it is actual using. I am not sure if anyone else has any experience with this. I really hope we can get it working though and am happy to help in any way.

from arlo.

I dove into the android apk and found these private key and certs. Perhaps someone can use them to start the connection? Hopefully this is helpful.
certs.zip

from arlo.

Hey @njschwartz, thanks for your help with this! I spent a little time the other day dorking around with rtmpdump, but didn't have much luck. If I get some time one evening this week I'll spend a little more time playing around with the certs you sent.

from arlo.

@jeffreydwalter Thanks a lot for looking. It isn't something crazy important, but it sure would be nice to be able to open the stream from a home automation dashboard instead of having to log into the app. I also tried rtmpdump and had zero luck...nothing I attempted got any output. I would think with the certs and keyfile it has to be doable, I just don't understand SSL well enough to do it myself.

from arlo.

@njschwartz, didn't have time today, but check this link out. It does a pretty good job of explaining what all of the parameters are in the json that's embedded in the tag for the rtmps player in the Arlo website.

It might be worth taking the values from the Arlo website and plugging them into the flowplayer just to see if it works. If it does, we might need to see if we can find the source to or reverse-engineer the flowplayer.

After you log into arlo.netgear.com, you can download the two flowplayer plugins used by the arlo site here and here

from arlo.

FYI, the JSON I'm talking about looks something like this:

config={  
   "key":"#$47ccXXXXX8604dXXXXX",
   "debug":false,
   "log":{  
      "level":"error",
      "filter":"*"
   },
   "play":{  
      "opacity":0
   },
   "clip":{  
      "autoPlay":true,
      "autoBuffering":true,
      "url":"48B4597VD8FF5_XXXXXXXXXXXXX",
      "live":true,
      "provider":"influxis",
      "connectionProvider":"secure",
      "bufferLength":1
   },
   "plugins":{  
      "influxis":{  
         "url":"flowplayer/flowplayer.rtmp-3.2.13.swf",
         "proxyType":"best",
       "netConnectionUrl":"rtmps://vzwow163-z2-prod.vz.netgear.com:80/vzmodulelive?egressToken=1059c0a9_XXXX_4faf_XXXX_dd786d2XXXXX&userAgent=web&cameraId=XXXXX97VD8FF5_1XXXX72684236"
      },
      "secure":{  
         "url":"flowplayer/flowplayer.securestreaming-3.2.9.swf",
         "token":"PgtFXXXXXXIyWZsfejCohmXXXXXXXb8REbL3yZLOF6uSUIyFXXXXXXXYuLuZ5MA"
      },
      "controls":null
   },
   "playerId":"48B4597VXXXXX",
   "playlist":[  
      {  
         "autoPlay":true,
         "autoBuffering":true,
         "url":"48B4597VD8FF5_XXXXXXXXXXXXX",
         "live":true,
         "provider":"influxis",
         "connectionProvider":"secure",
         "bufferLength":1
      }
   ]
}

from arlo.

I actually did exactly that. I set up a flowplayer in a webpage and used everything identical to the flashvars but it does not play. When I hit play on the flowplayer it calls https://vzwow171-z2-prod.vz.netgear.com/open/1 (the vzwow part seems to be any number between 1-250 or so) and I get a connection refused error in the logs. I assume it is an SSL thing but I do not really know....

from arlo.

Also it is interesting that I am pretty certain from looking at what you pasted that the token is the same for everyone. The key I am relatively certain is just the flowplayer commercial license and is also the same for all. The biggest piece of the flashvars that I don't understand is the egressToken in the rtmps address. Not really sure what that is all about...

from arlo.

FYI there's a similar discussion in tchellomello/python-arlo#8 to figure out the Arlo streaming setup, so it would be useful to combine efforts. One of the commenters there noticed that with the right User-Agent it will provide an RTSP stream that appears to be used by the iOS app. RTSP is a more open format, so it might be easier to work with than the Flash-proprietary RTMP.

from arlo.

from arlo.

Any luck? I found that node solution and posted it in the other discussion awhile back. Haven't had time to look into it.

from arlo.

from arlo.

@deanmcguire that is awesome. Nice work! So do you need to do anything else special to play that rtsps stream with ffmpeg? I thought I had tried that before with no luck but I may have had something off. Basically you just requested the iOS version of the stream and passed it to ffmpeg to play, that's it? I can't wait to test this... Thanks for sharing!

from arlo.

@deanmcguire you're the man! Thank you for taking the time to figure that out. :) Would you be interested in making a PR?

from arlo.

Nice @deanmcguire! Good deal to finally get a fresh start on this.

from arlo.

@deanmcguire just tested things out. Works great! It looks like something changed between ffmpeg 3.2.2 and 3.4, because I was getting Operation not permitted when I ran your command with the older version.

from arlo.

Just merged @deanmcguire 's latest commit, so streaming should now work! Thanks again!

Currently, the library just provides the url for the stream and will do the pinging. You will still need to use ffmpeg or something similar to get the stream.

Going to close this issue now, please file a new one if there are any problems.

from arlo.

@deanmcguire do you mind sharing your HA/Appdeamon code? Just bought a pair of Arlo-cameras and want streamin to work in HA.

I'm new to git, so don't know if you can contact me here but I've the same username at HA forums.

from arlo.

I'm getting "Protocol not found" when trying to use ffmpeg for rtsps?

from arlo.

@deanmcguire I’m on 3.4.1 and I’m getting the “protocol not supported”. When I had 3.2 it said “operation not permitted”.

Initially I wanted to stream my arlo Q in HA and homekit, then I settled for just arm/disarm but even that’s too much of a work. Could you maybe help me how to reverse engineer the request I need to make? I’m a little bit new to the mitm-proxy thing but I do understand what you’re trying to extract and it does make sense.

As soon as I understand how to extract the request I need to make I’m sure I could make Arlo Q supported too.

Edit: I managed to install mitmproxy and used it a bit. Seems that I can't load the modes and thus see the request to arm/disarm the arlo Q. What are you guys using to extract the requests?
With mitmproxy I can see regular traffic in the web, and also within the arlo app. But it doesn't load the stream or the mode-state.

from arlo.

@deanmcguire
Thanks for your code, I’ll play with it when I’ve figured out how to make requests to the arlo Q.
I’m using rasbian and the source is ffmpegs homepage. Funny is, I inserted the rtsps-link in an ios app that stream cameras and got excactly the same message, thus I don’t think it is due to my setup. Hopefully I figure something out when I extract the requests.

I’m using the command you posted above and that was commited.

Thanks, I’ll use chrome instead...the mitx proxy didn’t reveal much but it was fun to set up and play with. I built it on pi3 and it’s basically a mitm-proxy acting like a router. Might have some fun with it later..

from arlo.

@deanmcguire

Just an update: I managed to arm/disarm the Arlo Q with my raspberry pi. I tested to pull the stream from my media server and it worked, I dowloaded a 3min long 1080p video.

There is some miner changes to do for making it to work with arlo Q. I’m off for new year celebration but after the holidays I’ll can put up the changes for arlo Q.

So I guess I'll have som fun with automating Arlo Q, my plan is to disarm/arm based on location..using Homekit rather than arlo's own app. Then record continuously to a local disk if it's trigger manually or by a motion.

EDIT:
Recompiled for raspberry pi and now it works with ffmpeg, anybody else having the problem I had try this:

git clone --depth=1 git://source.ffmpeg.org/ffmpeg.git
sudo apt-get install yasm libvpx. libx264.
sudo apt-get install libssl-dev
./configure --prefix=/usr --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-openssl --enable-omx-rpi
make -j4
sudo make install

from arlo.

@Ritte88 would you mind contributing your arloq device schema from the GetDevices() call? I'm building a wiki page with all of the various Arlo hardware device schemas. I only have the first generation arlo cameras, so it's been tough to make the library make sense for all of the various Arlo camera/basestation combos, especially arloq and arlo go.

Here's the wiki page, which includes a script you can use to generate the device schema with all of the sensitive values obscured.

Thanks!

from arlo.

@deanmcguire random question for you here... I just realized that the library is leaking Ping() reply messages, since Ping() calls Notify() and not NotifyAndGetResponse().

Since you're probably the heaviest user of the HandleEvents() method, I wanted to get your input about this issue.

There are a couple of ways to handle things, but I'm on the fence. Calling NotifyAndGetResponse() inside of Ping() feels like the right thing todo since it's really sending a message to the event stream and a reply is sent in response.
The problem is that if you call HandleEvents(), like your code does, then HandleEvents can possibly "steal" the response for Ping(), which would cause NotifyAndGetResponse() to eventually timeout.
One possible mitigation for this would be to have HandleEvents() check the message for "subscriptions" in the resource element of the event message. If it starts with "subscriptions", then I could just re-queue that message in hopes that Ping() eventually gets it.
Something like this:

            while basestation_id in self.event_streams and self.event_streams[basestation_id].connected:
                event = self.event_streams[basestation_id].Get(block=True, timeout=timeout)
                if event:
                    if event.get('resource').startswith('subscriptions'):
                        self.event_streams[basestation_id].queue.put(event)
                    else:
                        response = callback(self, basestation, event)
                        # NOTE: Not ideal, but this allows you to look for a specific event and break if you want to return it.
                        if response is not None:
                            return response

There are a couple of issues with this approach.

1. The event for Ping() might keep getting handled by HandleEvents() when it's re-queued, which could impact performance, and possibly lead to an infinite loop.
2. Someone might want to do something with the "subscriptions", but it would be put back into the queue before we allow someone to handle it.

So, my big question to you is... Do you foresee any need to handle Ping() reply events in your HandleEvents() callback?

from arlo.

I ended up doing this 5956b1f

I basically made HandleEvents() requeue any message with a resource type that starts with "subscription". I believe only subscribe and ping messages send that resource type.
To handle the infinite loop issue, I just added a hacky sleep() just after requeuing the message, which seems to work okay. I suppose there may be a potential timing issue there, but it worked okay in my limited testing. The worst case scenario is that my sleep is too short, and HandleEvent() will pull that message from the queue and requeue it a bunch of times, but it should eventually get handled up by Ping().

from arlo.

@deanmcguire do you happen to have any idea how the mobile app pauses live video streams? In the browser, it uses the flowplayer controls to pause/resume. It would be really nice to figure out a way to pause an ffmpeg stream.

from arlo.

@deanmcguire I'm getting a weird error message when trying the command you posted above.
ffmpeg -re -i 'rtsps://vzwow368-z2-prod.vz.netgear.com:443/vzmodulelive/ALPHANUMSTRING?egressToken=ALPHANUMSTRING' -acodec copy -vcodec copy test.mp4
yields
'rtsps://vzwow368-z2-prod.vz.netgear.com:443/vzmodulelive/ALPHANUMSTRING?egressToken=ALPHANUMSTRING': Invalid argument

tried this command too:
ffplay -max_delay 500000 -rtsp_transport tcp rtsps://vzwow37-z2-prod.vz.netgear.com:443/vzmodulelive/ALPHANUMSTRING?egressToken=ALPHANUMSTRING&userAgent=Android&cameraId=ALPHANUMSTRING
but I get a 403 Forbidden (access denied)

might you know what I'm doing wrong?

from arlo.

Check the version of ffmpeg you're running. I just tested on ffmpeg version 3.4.2 and it works fine.

from arlo.

@jeffreydwalter Nevermind! Thanks! It was the quotes. "..." instead of '...' :)

from arlo.

ah, heh. I was looking at that, but thought nah... Glad you figured it out!

from arlo.

It looks like Arlo has changed things since this topic was last visited.
ffmpeg -re -i 'rtsps://' -acodec copy -vcodec copy test.mp4
now returns a 403 error so it seems they are now looking for additions authentication information. Does anyone know what additional information is needed?

from arlo.

@jbusfield can you please provide some helpful details? The output of ffmpeg would be a good start.

from arlo.

Users-MBP:Python jbusfield$ ffmpeg -re -i 'rtsps://vzwow18-z2-prod.ar.arlo.com:443/vzmodulelive/XXXXXXXXXX_XXXXXXXXXX?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXX&userAgent=iOS&cameraId=XXXXXXXXXX_XXXXXXXXXX' -t 10 -acodec copy -vcodec copy text.mp4
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with Apple clang version 11.0.0 (clang-1100.0.33.17)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
[rtsp @ 0x7f8b8c00a400] method DESCRIBE failed: 403 Forbidden
rtsps://vzwow18-z2-prod.ar.arlo.com:443/vzmodulelive/XXXXXXXXXX_XXXXXXXXXX?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXX&userAgent=iOS&cameraId=XXXXXXXXXX_XXXXXXXXXX: Server returned 403 Forbidden (access denied)

from arlo.

from arlo.

Yes I am using your library to run this code

try:
arlo = Arlo(USERNAME, PASSWORD)
cameras = arlo.GetDevices('camera')
basestations = arlo.GetDevices('basestation')
arlo.Subscribe(basestations[0])
url = arlo.StartStream(basestations[0], cameras[0])

print(url)
    
call(['ffmpeg', '-re', '-i', url, '-t', '10', '-acodec', 'copy', '-vcodec', 'copy', 'test.mp4'])

except Exception as e:
print(e)

The code prints this out:
rtsps://vzwow632-z2-prod.ar.arlo.com:443/vzmodulelive/XXXXXXXXXX_XXXXXXXXXX?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXX&userAgent=iOS&cameraId=XXXXXXXXXX_XXXXXXXXXX
[Errno 2] No such file or directory: 'ffmpeg'

I don't know why it cant run ffmpeg from code but rather than figure that out I instead copied and pasted the url into a command line ffmpeg

from arlo.

What version of ffmpeg are you using? Also, try the ffmpeg command with rtsp instead of rtsps.

from arlo.

ffmpeg version is 4.2.2

Using rtsp the output from ffmpeg is
rtsp://vzwow824-z2-prod.ar.arlo.com:443/vzmodulelive/XXXXXXXXXX_XXXXXXXXXX?egressToken=XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXX&userAgent=iOS&cameraId=XXXXXXXXXX_XXXXXXXXXX: Invalid data found when processing input

from arlo.

I have not found a solution for the issue

from arlo.

do we get the solution for this ffmpeg code I am getting the same error: Invalid data found when processing input.

it will be helpful if someone redirects me to somewhere else well if this is resolved.

from arlo.