Connection issue with the August Lock Pro gen3

[aeozyalcin]

Hello,

I love what you did with the python interface, unfortunately I couldn't get it to work on my end. I got the offline key and index with a rooted android phone, however when I issue the command "sudo python3 cli.py front --unlock", I get the following error messages:

Writing command: xxxxxxxxxxxx
Encrypted command: yyyyyyyyyyyyy
Traceback (most recent call last):
File "cli.py", line 38, in
lock.connect()
File "/home/pi/augustpy/augustpy/lock.py", line 55, in connect
response = self.secure_session.execute(cmd)
File "/home/pi/augustpy/augustpy/session.py", line 96, in execute
return self._write(command)
File "/home/pi/augustpy/augustpy/session.py", line 89, in _write
self.peripheral.waitForNotifications(10) is False:
File "/usr/local/lib/python3.7/dist-packages/bluepy/btle.py", line 560, in waitForNotifications
resp = self._getResp(['ntfy','ind'], timeout)
File "/usr/local/lib/python3.7/dist-packages/bluepy/btle.py", line 407, in _getResp
resp = self._waitResp(wantType + ['ntfy', 'ind'], timeout)
File "/usr/local/lib/python3.7/dist-packages/bluepy/btle.py", line 362, in _waitResp
raise BTLEDisconnectError("Device disconnected", resp)
bluepy.btle.BTLEDisconnectError: Device disconnected

Any clue what might be going on? Thanks!

[aeozyalcin]

I actually have it boiled down to "self.peripheral.waitForNotifications(10)". Bluepy seems to think that the lock is maybe disconnecting while it's waiting for a reply. This is happening during the first characteristic write Send SEC_LOCK_TO_MOBILE_KEY_EXCHANGE. I don't know enough about the interface to tell if it's an issue with encryption, or if it's related to the newer locks?

Any idea what might be the issue? I know I have the right offline key/index (obtained via rooted Android phone). I am really hoping you can help!

[aeozyalcin]

Some more updates, I dug deeper with debugging the BLE connection using the nRF connect app. When I connect to the lock with the app, and attempt to write something to the secure write characteristic of the lock, the lock will initially disconnect from BLE. This is what I think I am seeing exactly happening when using the augustpy library as well., which is a bummer. I am wondering if this a Lock Pro 3rd gen specific behavior. I would really love to see this work. Any suggestions are welcome.

[aeozyalcin]

Another update, the issue ended up being due to the offline key I was using. It didn't like my offline key at index 0 for some reason. When I switched to using girlfriend's key, I actually got it to work. I am really excited about the potential that this python library offers. However, one snag I am hitting is that when the connection is interrupted unexpectedly for some reason, next time I try to connect to the lock via python fails. I noticed that I have to then connect using the app on the phone once, which seems to "reset" something in the lock, and then I can connect again using the python library. Now onto figuring that bit out :/

[aeozyalcin]

I got it!!! The connection is rock solid now, I can connect and operate the lock any way I lock each time I want. I noticed that @Friendly0Fire mentioned about unreliable connection in the last commit, I think I figured out the issue.

So the issue was that I was randomly able to connect just fine and operate the lock. But then the next day when I try to connect, I would write the handshake key to the lock but receive no response back from the lock. I figured out that if I connect once using my phone with the app, then I was able to get the RPi to connect to the lock just fine without problems for a while, but then eventually my RPi would stop being able to connect.

It turns out, that during the connection process, we are not subscribing for indications from the SEC or the MCU. So what was happenning was, we write the handshake key, the lock updates the read register with its response, however BluePY was not getting the message because we hadn't subscribed for the indications. And hence why connection was failing. Furthermore, the reason why connecting with my phone made my RPi work for a while was that the phone was enabling all the indications on all the UUIDs on the lock, so when I connected with my RPi later, I was getting the indications. Until the lock turned off all notifications/indications. After that point when I try to connect and write the handshake key, I wasn't getting anything back.

Long story short, here is how you fix it:

                response = self.peripheral.writeCharacteristic(26, b'\x02\x00', withResponse=True)
                print("Subscription SEC request response: ",response)

                response = self.peripheral.writeCharacteristic(21, b'\x02\x00', withResponse=True)
                print("Subscription MCU request response: ",response)

These lines needs to go into lock.py inside the connect function to line 47, right after we retreive device characteristics, and before we set the sessions key. First line enables indications on the SEC, and the second line enables indications on the MCU.

I am so glad I got this figured out... I have spent all week on this and now this library will allow me to automate my home just like I wanted it. 😃

[Friendly0Fire]

First, thank you for pinging me directly. For whatever reason, Github hadn't auto-watched this repo for me, so I wasn't getting notifications. Sorry about that!

Second, that's amazing news, I'm glad you figured it out! I'll merge this change ASAP. I'd sort of given up on this project with how unreliable and inconsistent the connection was, so hopefully that'll be the change it needed!

[aeozyalcin]

No problem! Thank you for putting together the library. Please do let me know if this fixes your unreliable connection issue.

I have a 3rd gen lock, not sure which gen you have. The characteristic handles I am using are hard coded (26 and 21) and may not be the same for you. I initially tried to derive the handles dynamically by searching for descriptors under the characteristics, but BluePy was taking too long, and the lock was timing out and disconnecting (5 seconds of inactivity causes the lock to terminate connection). So to speed it up, the handles are currently hard coded. Let me know how it goes!

[aeozyalcin]

@Friendly0Fire tagging you in case you didn't see my last message. Thanks.

[benaranguren]

@aeozyalcin how did you find the char handles?

[aeozyalcin]

@aeozyalcin how did you find the char handles?

            self.command_service = self.peripheral.getServiceByUUID(self.COMMAND_SERVICE_UUID) 
            characteristics = self.command_service.getCharacteristics()
            #descs = characteristic.getDescriptors()
            for characteristic in characteristics:
                if characteristic.uuid == self.WRITE_CHARACTERISTIC:
                    self.session.set_write(characteristic)
                elif characteristic.uuid == self.READ_CHARACTERISTIC:
                    self.session.set_read(characteristic)
                    #descs = characteristic.getDescriptors()
                    #for desc in descs:
                        #print("found  desc: " + str(desc.uuid))
                        #str_uuid = str(desc.uuid).lower()
                        #if str_uuid.startswith("00002902"):
                            #mcu_sub_handle = desc.handle
                            #mcu_sub_handle = 21
                            #print("*** Found MCU subscribe handle: " + str(mcu_sub_handle))
                elif characteristic.uuid == self.SECURE_WRITE_CHARACTERISTIC:
                    self.secure_session.set_write(characteristic)
                    print("Set Secure Write")
                elif characteristic.uuid == self.SECURE_READ_CHARACTERISTIC:
                    self.secure_session.set_read(characteristic)
                    print("Set Secure Read")
                    #descs = characteristic.getDescriptors()
                    #for desc in descs:
                        #print("found  desc: " + str(desc.uuid))
                        #str_uuid = str(desc.uuid).lower()
                        #if str_uuid.startswith("00002902"):
                            #sec_sub_handle = desc.handle
                            #sec_sub_handle = 26
                            #print("*** Found SEC subscribe handle: " + str(sub_handle))

Here is the code I wrote to find the handles, it goes into the lock.connect() function. Once I determined it was taking too long for my liking, I commented out most of it, and used hard coded in the notification subscription requests. Hardcoded values work to this date. You will need to uncomment most lines, clean it up a bit and it should print out the handles for you. Let me know how it goes, good luck!

[benaranguren]

Thanks @aeozyalcin. It's been working solid for me too with your patch. I'm not familiar with BLE, so how did you know to look for 00002902?

[benaranguren]

Oh wait ... I see in nRF app that 0x2902 is listed as Client Characteristic Configuration UUID: 0x2902. That must be the link.

[aeozyalcin]

@benaranguren yep you got it. 00002902 has some sort of significance in BLE, the readable/indicatable characteristics typically have that descriptor.

That's awesome, I am glad to hear I've contributed! This project taught me a lot about BLE, very fun and rewarding. I have made many more modifications to FireFly's library, and I hope to share it all one day once I finish the "grand" project that I have been working on.

[benaranguren]

One more question @aeozyalcin . Where did you get b'\x02\x00' ?

[aeozyalcin]

@benaranguren if you look at nRF Connect debug output as you have the August app running, that's what we are writing initially to the characteristic to subscribe to indications. I think 02 is indications, and 01 is notifications (might be the other way around).

[benaranguren]

Pretty cool. Thanks.