Tutorial

[Albatros3]

Hi and a big hello from france.
Great job you're done.
Have you got a tutorial or a link for schemas and installations please?
Thank you.

[dridri]

I just wrote the beginning of the first draft : https://github.com/dridri/bcflight/wiki/Building
I hope it will help and will continue it as soon as possible.
Tell me if you have any questions :D

[smarkoco]

Amazing work!! Would love to get this drone setup integrated with my planned "Control a Drone Over the Internet" concept for a web application (eventually on cloudchop.net).

Are there perhaps any updated tutorials or list of dependencies/requirements for building? Would also be great if there were a couple brief explanations of the overall information flows, for example, I was not sure about how the control signals are sent, guessing the LoRa Radio? Do you think it's possible to receive a buffer of frames on a desktop and forward them to a remote browser, and also receive flight commands from that same browser over a bidirectional socket (e.g. with Tornado) or would the closed-loop latencies be too excrutiating?

Was also wondering what drone build you used or would recommend?
Thanks so much!

[dridri]

Hello, thanks for your interest in this project !

As you could see on the latest commits, I'm getting my hands on this project after a very long break and now working to make it more reliable. There's indeed a lot of work on documentation needed, this should grow in the next weeks.

The current readme is unclear about radio, you can actually use any kind of link you want : it's currently possible to use some FSK/LoRa modules, UDP/IP over regular wifi, and raw wifi using compatible dongles (it sends raw packets without need to associate with ground station). It's also possible to handle other radio communication systems by inheriting the Link class and overloading the few virtual functions it exposes.

The video feed can be sent back via legacy analog VTX with ~50ms glass-to-glass latency, but also via raw-wifi / regular-wifi with an extra cost of ~30ms and ~50ms latency respectively.

When I started this project I only used the wifi/raw-wifi to send and receive both control signals and video feedback, the code is dusty and needs a massive rewrite but what you are asking for is totally possible with 200~250ms total latency which should be just Ok if you intend to fly it smoothly.
The overall system could be something like : client browser → websocket → server & ground-station → wifi/raw-wifi → drone
The libcontroller is intended to be used in this kind of project, it has just the minimum required code to communicate with the drone from ground and should be easy to include in any program.

My current setup uses the carrier board described in the Hardware section with some 20$ ESCs and motors, a TrampHV VTX directly connected to Raspberry Pi's composite video output and the newest Raspberry Pi Camera Module 3, nothing more.

TL;DR : what you are asking for is possible but not in a reliable way for the moment as the code is old and faulty, but I hope to progress quickly during the next weeks

[smarkoco]

Thanks so much for the in depth reply, and happy Sunday! no worries at all about the documentation, I can see you've already put in a ton of work, and when a project is still in flux it can just be too time-consuming to constantly update documentation lol

Luckily I was able to get the controller_pc module to build and run last night on my Ubuntu Studio 22.04 system! Just had to install a few packages with apt when the make command threw some helpful "module not found" errors (they were usually related to *-dev packages for wireless, etc.).

For some reason, the below commands from the files HERE were not able to find that package even though I eventually explicitly added the bcflight/external path to my include path (??)

extern "C" {
#include <codec_api.h>
}

so I had to replace with the below to get it working

extern "C" {
#include "../../external/openh264-master/codec/api/wels/codec_api.h"
}

Dependency installs:
I also just ended up building and installing any other missing packages from github repos.

1. Qt:
   • sudo apt-get install qtmultimedia5-dev qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools
2. nasm:
   • sudo apt install nasm
3. QScintilla: Download source from HERE. Then run:
   • tar -xzf QScintilla_src-2.13.4.tar.gz
   • cd QScintilla_src-2.13.4/src
   • qmake
   • make
   • make install
4. MP4V2: repo located HERE
   • git clone https://github.com/enzo1982/mp4v2.git
   • cd mp4v2
   • cmake . && make
   • make install
5. Shine: It was a little tricky for me to find the right "shine" repo HERE
   • git clone https://github.com/toots/shine
   • cd shine
   • autoreconf --install --force
   • automake
   • ./configure
   • make
   • make install

After these installations, your Find*.cmake scripts in bcflight/controller_pc/CMakeModules worked perfectly 👍:
- git clone https://github.com/dridri/bcflight.git
- cd bcflight/controller_pc
- cmake -DCMAKE_BUILD_TYPE=Release -S <git clone path here>/bcflight/controller_pc -B <git clone path here>/bcflight/controller_pc/build
- cd build
- make -j4
- ./controller_pc

🎉

Next Steps:
- I just ordered the camera and compute board, and will try getting the video feed working first.
- Sadly the BMP180 and BMP280 are now "obsolete", and none of their other models have the 8SMD package, so I'll probably delay getting that piece. Seems like they upgraded to either Bosch BMP390 or BMP581
- ordering ICM-42605
- I'll probably order your board from OSHPark.com (looks like what you did from the purple board 😁 )
- What would you recommend as a ground station? Or can my wifi interface just communicate directly with the RFM95W for initial testing (sorry kinda new to the drone world)
- Finally I'm having a little trouble figuring out which pins are the "video composite out", would you mind letting me know? Are these to be accessed on your pcb board or the cm4 itself?

Thanks a bunch, will let you know how it all goes!

[dridri]

woow that's a nice work, would you mind if I add these steps into my readme/wiki ? That's the kind of thing I made so long ago that I forget which packages need to be installed 😅

About openh264 and mp4v2, I'm thinking about switching to ffmpeg (avformat/avcodec combo), this would reduce these external dependencies. libshine should have been available as an apt repo, but it will be dropped too when switching to ffmpeg.
Glad you have a Ubuntu machine, I never tested compiling it on Windows.

Erf, didn't see that the BMP180/280 were obsolete, I will take a look at which replacement has the longest lifespan.

Did order my board on JLCPCB, but close enough. I'll put the BoM somewhere as some parts could be tricky, like the camera connector that is a top-contact one (798-FH12A-22S0.5SH55).

The RFM95W uses a totally different communication protocol, and does not have enough bitrate to handle a video live stream. So you probably won't need any but instead use a WiFi USB dongle that supports monitor/promiscuous mode.
This is where we enter into the unknown : it's been years I didn't work on it but the principle is still ok and just need some rewrite (I currently use RFM95W for controls and analog VTX for video as it has the lowest latency). For best performance you will need wifi adapters that support monitor mode, this allows sending raw packets without the need to associate the stations. This kind of adapters : Best Kali Linux WIFI Adapter with Monitor Mode (something like the AWUS036AC on drone, and AWUS036ACH on ground).
But before spending money based on unknown result, I recommend that you just use regular wifi with embedded antenna on Raspberry's side and built-in on desktop side until everything works as expected. Using regular WiFi with TCP/UDP will limit you to ~100m / ~300ft distance but is well enough when prototyping.

Since you need the video feedback directly on your PC you won't need the composite output, but for informational purposes it's on pin 111 of the Compute Module 4 and is exposed on the jack plug on other models (image) (and via "TV" test-pad on PiZero, right under HDMI jack).

To sum up, this is what's missing to get your project done :

• A way to control the drone from PC, as you can see there's currently just a "Throttle" slider but nothing to control its attitude. I think the best solution would be to use an XBox / PS3 / PS4 controller, Linux happily handles them as generic joysticks that are very easy to implement. If you already have a regular drone controller, you could use it when prototyping as my board can handle S-BUS input.
• Update the video live stream code to use the new libcamera and v4l2 encoder (it's currently using OpenMAX legacy code that's going to be deprecated, and does not support the new camera modules)
• Get raw-wifi back on track (I'm going to take a look at how OpenHD did on their side, and probably try to make it directly compatible with them later)

I'll start on the libcamera/v4l2 combo, then manual control from PC, and finally raw-wifi

Speaking of OpenHD, if you need a working solution quick you probably have more chances with them aha.

[smarkoco]

Yeah for sure! that would be great to see it added to the Readme.md/Wiki, feel free to let me know if I should Pull Request it (then you could edit), whatever's easier

Would love to see ffmpeg get used, I love that package, I use it in my work in motion tracking/frame processing/video conversion, and a lot of the commands are highly parallelized

Thanks for pointing out the RPi camera connector, was actually searching eBay for something for that haha 😅, yes I was actually about to ask about all the essential connectors earlier, I think I may try to prototype a bit with the Regular RPi4B if I need to access more peripherals (USB/etc), luckily I do have one of those laying around the house

I'll definitely give the regular TCP/UDP route a try first, and will keep those other adapters in mind for the future. Thanks for all the tips!

Would love to start contributing if I can, I do have an xbox controller, and can definitely give that a try! I was also thinking of trying to map some keyboard input to flight control, such as how a helicopter is flown in Call of Duty, but that might be a bad idea for reasons I can't think of

(Edit: This could also map the mouse cursor changes onto some combination of Yaw/Pitch with some sort of transformation matrix, but that might be too complex)

[dridri]

Commit 25299a8 brings back video live stream feature from drone to PC controller app. I'll add a description into the wiki, but in the meantime if you're curious you can take a look at the config.lua file here and there.

With 8mbps bitrate there seem to be less than 60ms glass-to-glass latency over ethernet, libcamera+v4l2 is surprisingly good compared to old closed-source drivers and OpenMAX (was 120+ over wifi at only 2mbps)