Sharing drone data in Mapbox for fun and profit

Autonomous flying vehicles (like Site Scan!) capture all kinds of interesting GIS data–photos, projected photos, and flight logs, to name a few. Furthermore, drone imagery can be processed into other GIS data products like orthomosaic images and point clouds that become infinitely more valuable when combined with traditional GIS data sources like property boundaries and civil engineering diagrams. That said, sharing these data off-line is quite challenging. Massive geotiffs are painful to open in heavy GIS software like ArcMap and QGIS, individual images are troublesome to keep track of when not associated with a map, and flight logs are near useless in their raw .tlog or .bin form. Our friends at Mapbox have assembled an incredible (and free up to a substantial quota) toolkit for sharing all of these data in a simple webview format.

Follow these instructions to create an interactive map showcasing your drone data like the one below, whose source code is available at https://github.com/dmckinno/Mapbox. For this particular example, I overlaid a property boundary and fantastical building footprint kmls, an orthomosaic geotiff, and a handful of images viewable by clicking on the location at which they were taken, but essentially any GIS vector or raster data are sharable using Mapbox.

    1. Create an account over at https://www.mapbox.com/studio/signup/.
    2. Open Mapbox Studio. If you are not interested in adding interactivity to your maps, you can share all of your drone data without writing a single line of code. Simply upload your data a a tileset and add your tileset to a style.upload-tilesetMapbox accepts raster data in the geotiff format and vector data in the kml, which can be generated from Arducopter .tlogs using Mission Planner, gpx, which is the standard output from handheld GPS devices, GeoJSON, a GIS standard that is easily passed between systems, .shp, an open ESRI format, and .CSV formats. After uploading any of these file formats, Mapbox will slice the data into either vector or raster tiles that can be beautifully navigating on the web.Note you are unable to display a single nadir photo or a point cloud using Studio. To display a single nadir image in Mapbox, you must write a few lines of JavaScript (see below for examples). If you would like to display a point cloud, you must first use laslib or a commercial tool like ArcMap to convert to a geotiff.Once you have uploaded your tileset, create a new Style.styleChoose the desired basemap, add your tileset(s) to the map, and click “Publish.”mapbox-studioMapbox will take you to the “Preview, Develop, and Use.” Copy and paste the generated URL and send it to all of your friends. This particular example is available here.preview-and-develop
    3. Great! That’s the easy part. Static maps are nice, but Mapbox has a host of excellent examples that guide you in creating interactive maps displaying a wealth of different data types.First, the structure. Please clone my example repo at https://github.com/dmckinno/Mapbox to follow along. Tilesets uploaded via Mapbox Studio cannot be manipulated by the user viewing them in the browser. To build the types of interactivity that I showed in the example above, you must write a few lines of JavaScript. Fortunately, this is quite straightforward and nicely mirrors the GUI options in Studio. To add a layer, use

      map.on(‘load’, function () {
          map.addSource(‘ortho’, {
              type: ‘raster’,
              url: ‘mapbox://dmckinno.8u0goq8l’
          });
          map.addLayer({
              “id”: “ortho”,
              “type”: “raster”,
              “source”: “ortho”,
              “source-layer”: “OrthoImage-6x2l6d”,
          });
      });

      This is a combination of the map.on and map.addLayer functions. I don’t know enough about JavaScript to explain exactly why they are both needed, but they are always together. I believe a layer is created and then placed on the map. In this case, I am adding a raster from a given Mapbox URL (this is creating by concatenating mapbox:// and the Map ID shown inside the tileset view) and placing it on the map with the ID “ortho.”map-idI always keep the source and the ID the same, but I’m sure some Mapbox wizard understands why they may be different in certain cases. The source-layer comes from the individual layer within the tileset. The is irrevelevant for rasters, but important if a vector tileset contains several different features. Nonetheless, the source-layer is available in the “Raster tileset” or “Vector tileset” tab in the tileset view.

      source-layerVector layers are added in a similar fashion.

      map.on(‘load’, function () {
          map.addSource(‘views’, {
              type: ‘vector’,
              url: ‘mapbox://dmckinno.9erzotdj’
          });
          map.addLayer({
              “id”: “views”,
              “type”: “circle”,
              “source”: “views”,
              “source-layer”: “redhorse_photos-6besux”,
              “paint”: {
                  “circle-color”: “#ffffff”,
                  “circle-radius”: 3
              }
          });
      });

      The only differences arise in the options available for displaying the data beautifully. Vector data can be painted with different colors, thicknesses, fills, and opacities. Here, I simply drew white circles that indicate photo locations, but more complex vector layers can be displayed in infinitely complex and beautiful ways. The Mapbox documentation walks through the options in painstaking detail.

    4. Now you have your orthomosiac and raster data loaded, but you may want to augment the orthomosaic with individual nadir images. If you calculate the coordinates of the four corners of the image using the altitude of your drone and the focal length of the camera using simple geometry, you can display nadir images as psuedo-orthomosaics using the code below.

      map.on(‘load’, function () {
          map.addSource(‘photo1’, {
              type: ‘image’,
              url: ‘http://www.ddmckinnon.com/wp-content/uploads/2016/09/DJI_0202.jpg’,
              coordinates: [
                  [-105.2311759, 40.0848768],
                  [-105.2335889, 40.0851527],
                  [-105.2333181, 40.0865392],
                  [-105.2309051, 40.0862633]
              ]
              });
          map.addLayer({
              “id”: “photo1”,
              “type”: “raster”,
              “source”: “photo1”,
          });
      });

      It is somewhat challenging to embed two Mapbox maps in a single WordPress post (I use Code Embed, but please let me know if a better way exists), but an interactive map is available here and an image preview below. Note that because the image is not tiled, it is much less responsive than a true tiled orthomosaic.nadir-image

    5. The final vector layer type you may be interested in displaying is a image locations. It is often helpful to show where images were taken on a map and be able to view them based on context, whether project appropriately on the basemap or not. To do this, you must create a .kml, .shp, or .geojson file from image EXIF data. If you only have a few images, you can easily do this manually, but if I would recommend using the exif-to-geojson tool for more than a handful. This tool worked beautifully for the fifty or so photos I posted from my off-road motorcycle trip from Boulder to Moab. Once you have the geojson file containing all of the photo information, add a description field with the image URL and any text you like to each image. You can see another nice example of this in the map of my motorcycle trip.

      “data”: {
          “type”: “FeatureCollection”,
              “features”: [{
                  “type”: “Feature”,
                       “properties”: {
                          “description”: “<img src=”YOUR IMAGE URL HERE”> and any text or                       links you like”
                      },
                      “geometry”: {
                          “type”: “Point”,
                          “coordinates”: [YOUR LONGITUDE, YOUR LATITUDE]
                      }
              }]
      }

    6. Now that you’ve added all of your layers, you likely want to include some interactivity. I’ve played with all of the Mapbox examples, but most relevant here is the ability to show and hide layers and show images upon a click. For my example above, I copied and pasted from Mapbox with minor modifications and I recommend you do the same, making sure that you grab every line that you need.
    7. And voila! Now you have all the tools you need to share your drone data with all of your friends and followers using Mapbox and a few lines of JavaScript.

 

October Site Scan R10C media roundup

Two months after launch, Site Scan R10C continues to get phenomenal press coverage throughout the tech, engineering, construction, and UAV publications. In October, I had the opportunity to sit down with Luke Geiver of UAS MagazineElizabeth Manning of Construction Business Owner, Jeremiah Karpowicz of Commercial UAV News, and Tanya
Weaver of Develop3D Magazine to share my thoughts about Site Scan, the passage of Part 107, and the potential of drones to revolution the $8T construction industry. Have a read by following the links below.

3DR Enterprise manager explains efforts for commercial markets in UAS Magazine

Helping Drone Operators Get the Most Out of a Commercial UAV Investment in Commercial UAV News

Expert insight into the new FAA drone regulations in Construction Business Owner

Eye in the Sky: 3DR takes off with new Site Scan solution in Develop3D (.pdf link or browse below)

Develop3D inexplicably does not post the text of their articles online, so searchable full-text is below.

Eye in the Sky: 3DR takes off with new Site Scan Solution

“Literally, one button to measure your world and then all the data is seamlessly pulled into our cloud and the Autodesk cloud so you can manage it” –Dan McKinnon, 3DR’s director of product

It’s predicted that by 2024 the market for drones will hit $91 billion. Tanya Weaver chats to 3D Robotics, one of the many drone companies making a grab for the open skies, about its move into the enterprise market with a new ‘drone to cloud solution’ called Site Scan.

solo-r10c

The Solo drone collects high-quality data with the Sony UMC-R10C camera.

Once a plaything of the hobbyist, drones have become increasingly sophisticated in recent years, making these unmanned aerial vehicles (UAVs) appealing to the professional user. And one drone company that has its eye firmly set on the enterprise market is San Francisco-based 3D Robotics (3DR), arguably one of the oldest drone manufacturers worldwide. This is somewhat absurd as 3DR is less than ten years old but, as we all know, technology in this market moves fast and the company has come a long way in that time.

HOW IT ALL BEGAN

3DR started out in 2007 making UAV kits for the DIY community and soon progressed into creating an open source flight code, which was used to build Pixhawk in 2012. This autopilot system is today running in more than 750,000 autonomous vehicles. In 2014 3DR made a foray into developing its own consumer UAVs with the launch of Iris, soon followed in 2015 by Solo, a powerful quadcopter designed with the help of San Francisco industrial design firm Astro that delivers high-definition video from its onboard GoPro camera straight to an app on the user’s mobile device. Then, in a quest to move Solo’s usefulness from the consumer into the commercial space, earlier this year 3DR launched its aerial analytics platform Site Scan.

ON THE ROADMAP

3DR’s CEO, Chris Anderson, claims that it was always on the roadmap to move into the enterprise market and that the consumer applications for Solo were just the start. “I was hired around a year and a half ago with the mandate to build our first enterprise product,” says Dan McKinnon, director of product at 3DR, who himself is no stranger to drones having built an agricultural drone surveying company, Agribotix, with his father whilst at graduate school. McKinnon initially spent a couple of months with potential customers gaining understanding of the issues they had on site and how these could be solved with drones. For instance, moving earth around on construction sites is a difficult and often hazardous endeavour. Traditionally, a laser scanner would be used to determine how much earth needed to be moved, but instead a drone can collect all this data conveniently from above. McKinnon soon realised that a camera-to-app-to-cloud system would be useful to those in construction as well as other industries such as telecoms, energy, infrastructures and utilities who have a need to model, inspect, survey or scan projects. So, he went about putting a team together to create an enterprise offering. The resulting Site Scan solution, which was created in partnership with Autodesk, takes images from a job site via the Solo, which is controlled with a few taps and swipes of an app on a tablet (currently, only certified for use with Sony Xperia tablet). The captured geotagged data is then uploaded to the 3DR cloud where Autodesk’s ReCap reality capture and point cloud software then processes it into an image of the user’s choice, whether that be orthorectified 2D maps, 3D renderings, mosaics, point clouds or 3D mesh. Once processed, this reality data is available on the Autodesk A360 cloud platform from where it can be downloaded and used by Autodesk software such as BIM 360, InfraWorks 360, AutoCAD or indeed other software tools. According to 3DR, the process of capturing, tagging, uploading, processing and importing data, depending on the area, is generally a couple of hours. “We wanted the product to be so easy and intuitive to use that a five year old could do it and I think we were successful. Literally, one button to measure your world and then all the data is seamlessly pulled into our cloud and the Autodesk cloud so you can manage it and generate nice data products like reports or 3D models,” says McKinnon.

diyd

The Solo drone was flying with the GoPro camera however, Dan McKinnon was keen to use a new QX-1 camera from Sony and so prototyped a contraption that connects this camera via software and hard cable to the Solo.

NEW SOLUTION

However, McKinnon recognised that although the GoPro is heralded as the world’s best action camera, not all commercial customers would be satisfied with the quality of the images it takes. “The GoPro is a great camera but there are a lot of limitations with it for professional use. Extremely high image quality is paramount because the sharper the images, the better our customers are able to monitor their sites,” comments McKinnon. He had a fondness for the recently launched Sony QX-1, which despite being compact and lightweight (180g) has a large 20 mega pixel APS-C image sensor that is 15 times larger than the GoPro and features an exceptional signal to noise ratio to deliver superior image quality in all light conditions. “I took one of these QX-1 cameras with the Solo and went into the garage on the weekend and prototyped a crazy contraption that connects the camera via software and hard cable to our drone. “This feat of engineering contains no fewer than two Pixhawks, six batteries, 20+ feet of cable, two companion computers, five ESCs, 10+ feet of strapping tape, two transmitters and a RC Timer CM102 camera gimbal,” smiles McKinnon. He photographed the prototype and wrote a post on DIYDrones.com, which was in fact started by 3DR’s CEO in 2007 and has since grown into the world’s largest open robotics development community. Having received a positive response from the community, he ran the idea past a few customers, who also responded positively, before deciding to pursue it by contacting Sony directly.

GETTING SONY ONBOARD

John Monti, director of Visual Imaging Solutions at Sony Electronics, was impressed and arranged for McKinnon to have a face-to-face meeting with the top executive in Sony’s camera engineering organisation, who was visiting an unrelated trade show in the US. “During the presentation and with the conference table literally covered with circuit boards, cables, lenses, etc. Dan patiently explained what was needed to create a simple and elegant solution. In the end, the gambit left a strong impression on Sony’s executive as well as the team of key marketing and planning folks and the meeting literally changed Sony’s thinking on the market potential for commercial drone cameras,” says Monti. “It was quite a surreal experience for this top guy from Sony to see me pull stuff out of the case to see what monstrous contraption I’d created to integrate this off the shelf camera with our drone,” laughs McKinnon. Following a visit to the Site Scan team at 3DR’s facilities, the Sony team responded a few weeks later with a proposal that met the requirements without having to develop a totally new product from the ground up. “They built us a custom version called R10C and wrote some special software. We then wrote our own software and built a stabilisation gimbal for the camera,” says McKinnon. According to Monti, the creation of the two axis gimbal would take considerable mechanical design expertise since the Solo will fly with two different Sony lenses, one of which has a remotely-controlled power optical zoom function, changing its centre of mass and physical size as it zooms in and out.

BRUSHLESS VS SERVO MOTORS

Matt Ambauen was the lead mechanical engineer on the project and hit the ground running as the timeline was extremely tight. “We initially had planned to do a gimbal based around DC brushless motors, similar to the product we had on the market already, but the weight of the brushless motors would have pushed the stabilisation system over the maximum payload,” he claims. The R10C is not a light unit and its centre of gravity actually changes pretty drastically, especially with the zoom lens. As brushless gimbals are very sensitive to balanced payloads, Ambauen decided to resurrect an older technology in the form of high-end servo motors. These motors are coupled to a sensor for position feedback so although it may not be ideal for delivering high quality video, it is more than capable of stabilising the camera stills, which is what the Site Scan user would be interested in anyway. “Servos are not as smooth as the DC brushless motors would have been – it’s a little jerkier than what we would have liked but since we were taking stills as opposed to video, it was a concession we were willing to make. Any jitter in the gimbal would be really slow relative to the fast frame rate of the camera,” comments Ambauen. The benefit of using servo motors was also that the whole system would be lighter as it wouldn’t need any additional components like motor control boards or encoders that would be needed if a brushless gimbal was used. Of course, weight is a major consideration in the development of drones because any excess can cause them to perform negatively. “There is an autopilot that is doing the stabilisation and trying to keep the vehicle in the air and if it doesn’t have the control forces needed to do that, you start to drift all over the place. The camera is actually quite heavy and was approaching the maximum payload for the vehicle so we had to keep the gimbal component of it as light as possible so we didn’t go past the controllable payload for the quadcopter,” explains Ambauen.

in-field

Civil engineering and design firm Kimley-Horn used Site Scan recently to scan the Red Rocks Amphitheatre in Colorado, which resulted in a highly detailed and sharp 3D model of the site. The Sony UMC-R10C is available now, completely integrated with the Site Scan app on iOS. The total package is available for $12,200 including the drone, camera, software and full support.

GIMBAL DESIGN PROCESS

Having decided on the use of servo motors, Ambauen kicked off the gimbal’s design process by importing the step files that Sony had provided into SolidWorks. “We only had to interface with a couple of faces of the camera and I spent a reasonable amount of time trying to design these interfaces directly with the camera off the CAD. The geometry wasn’t a 100 per cent correct so we had to iterate and used 3D printing a lot for that. “We would tweak some dimensions and print a new piece to see how it fits against the camera and then try again if it wasn’t quite right. It took four or five iterations to dial it in,” describes Ambauen. Meanwhile, the 3DR software team had been working on the software development side of things and at the start of September 2016 the company announced the availability of Site Scan as an integrated bundle with Sony’s UMCR10C camera. “I’m really proud of the work we have done with Sony and the team to build the best flying camera in the sky, it’s well above anything the competition is offering. On top of that we are offering unparalleled customer service with a dedicated support team available to guide our customers through the Site Scan process and to be on hand to answer any questions,” says McKinnon. Although it’s still early days, 3DR has racked up a number of Site Scan customers already (see box on page 26), and with other industries seeing the applicability of drones to what they do, this is only going to increase.