MapJack

The spring of 2006 Google Maps had begun to make a great impact on people with its detailed satellite imagery. People all around the world would zoom into their neighborhood to find their house, and do virtual tourism, looking at landmarks like the Michael Jackson mansion and Guantanamo Bay Naval Base.

I had recently moved to Thailand, and was looking for something creative to do. As I played around with the satellite imagery, it struck me that the next cool thing would be detailed photographs from every street on the map. The idea was to transition down from a top-down view to a street view, for any location in the city. The daunting task of collecting all this imagery made me put away the idea for a few months.

Solving technical challenges

But the same summer I was determined to make it work, because I had a feeling that I was onto something big. I figured that I could mount a DSLR camera on my scooter, and then drive down and take pictures of every street of Chiang Mai, where I lived at the time.  

My neighbor Rong Daeng (uncle red) was a mechanic, and helped me weld together a mount for my scooter. After a few trials with tilted and shaky shots, I tried a different approach to wear a backpack with a custom camera mount on it instead.

Below: Rong Daeng's workshop and to the right posing with his daughter.

It took some practice to drive upright and steady with the scooter, but it worked well. Images came out sharp and level.

Below: The custom backpack and my scooter, a Honda Airblade.

Next thing was to figure out how take shots automatically, so I didn't have to push the camera button while I was riding.

I constructed a triggering system, from parts of an old bicycle trip-meter. A small magnet was mounted to a spoke on the front scooter wheel, and a reed-switch mounted on the fork sensed the magnet as it passed by every revolution of the wheel. The switch connected to an electronic box with a counter, which in turn triggered the camera at a certain count. It was MacGyver tech, right from the start.

Below: The trigger box. It was designed on a CMOS 4026 digital circuit, that counted signals to show the count on a display, and you could preset the count where it would restart again, using two knobs. I kept an on/off switch in my pocket, and turned the device on as I entered a new street. This system paced out the shots at an even distance, no matter how fast I was riding. 

Maurycy Siedlowski, owner of the local store Eagle GPS, supplied a small GPS logger, and over the years he became a close friend of mine. I would keep the logger with me while I was riding to record the coordinates of the pictures.

Next step was to create the production software to process the collected data. This was the biggest challenge, due to the thousands of pictures collected. Luckily Adobe Photoshop had an API, so I wrote an image processing pipeline for it. The images ran through it one by one, correcting for lens effect, adjusting white balance, colors, and sharpness, and finally resized, cropped and saved at a small file size.

The main production tool was the map view and dot alignment tool. It was written in VS2005 and C#. It downloaded the GPS data for the streets ridden, and matched it up with the time stamps from the picture file EXIF data. This would show as dots on the screen, overlaid on a satellite map that I downloaded from Google Maps. The precision of the GPS data was far from enough, so every dot had to be manually aligned with the street.

The trigger box kept an even distance between the shots, and this fact could be exploited to make the dot alignment quick and easy. You simply dragged the dot at the start and end of the street to their correct locations on the map, and all dots in between fell into place automatically. For curved streets you also adjusted a few dots in between. In theory this was easy, but the algorithms to make this work were very complicated.

Next step was to create the web site to expose the images. I decided to call it "Show The City" and it was launched in November 2006, see screenshot below.

In the middle is the street view picture, and below is the map that indicates the position of the picture. I had information texts for all major attractions, and for a few I also had video introductions, which I made with my friend Richie Castro, who later became an integral part of MapJack. At the top was a directory of city attractions. Clicking the arrows in the picture would move you up and down the street, step by step. This was the culmination of six months of hard work, and I felt pretty proud of how it turned out.

I emailed a few friends to get some feedback. Peter Ekelöf, an old friend from the time at WM-data, liked it and passed it on to his American friend Randy Segler, who in turn got very excited. The three of us decided to take it to the next level, and create a company to try and make some money from it.

We agreed to map some major US city to make a big impression in the media. Our first choice was to do all of Manhattan, but since it was in the middle of winter, we picked San Francisco instead.

Randy acted as angel investor, putting in the money to the get the company started and to map San Francisco. I became the CTO and Peter the CEO, with his background in economics and project management. Randy came up with the name MapJack, which we all thought sounded cool. The small figure in the map became our mascot Jack.

Back to the drawing board

This was early January 2007, and I went back to the drawing board to improve the system. Instead of scooters, we would use cars, and the camera units would pack four cameras instead of one, pointing front, back, right and left. This way we could get a better view of the shops, and only have to drive each street once. But the main change was that we carried a laptop computer in the car, which ran a custom software that controlled the mapping session. It had a map with dots for all mapped streets, and an icon to show the current location. You simply drove down the streets, turning on and off the mapping with a switch, and the software took care of the rest.

Below: Functional diagram of the mapping equipment.

Below: The camera unit had four Nikon cameras with wide angle fish-eye lenses.

The camera unit was made from a large aluminum pot found at the local market, with a reinforced bottom courtesy Rong Daeng. On top of the unit was a lid with the GPS logger taped to it. The whole unit was covered in heat reflective foil to make it cool and to look futuristic.

Each camera was connected through its remote shutter connector to a splitter box, which was just four opto-couplers, which can be seen in the middle of the camera unit. 

From that splitter box a cable ran down into the car and the control box. The camera unit did not have any feedback of shots being captured, so we placed a small microphone in there, connected to a loudspeaker in the car, so the driver could hear the clicking of the cameras. The four cameras were also connected to a USB hub, and then to a cable into the car, so pictures could be downloaded and inspected on the fly.

The control box was the interface to the laptop computer, so it could control the cameras. True to MacGyver, I found $5 game controllers in a local computer shop. I took out the PCB, reverse engineered the API, and added a digital counter and a debouncer circuit, plus added some opto-couplers to shield away interference from the car.

Below: The control box schematics. OUT went to the splitter box (cameras), SENS was connected to the reed switch mounted near the drive shaft of the car, USB was connected to the USB port of the computer.

The computer would read the four bit state of the counter, and at a certain count, trigger a pulse to the cameras to take a shot.

Mapping San Francisco

Mid March 2007 me and Richie traveled to San Francisco, and found an apartment to rent in Daly City, just 20 minutes outside the city. Richie's friend Brad Madrid helped buying two old reliable Volvos, and setting up the equipment. The cars had to be modified to fit the reed switches. A small magnet was epoxy glued to the drive shaft of the car and the reed switch mounted close by.

Below: Me and Richie busy with the camera unit. It was mounted to a regular tripod, and then strapped to the rack of the car.

Below: The production software was called PanoPal, and was used both for the capture of the pictures, and also the post processing. Dots (picture locations) were color coded to show the step in the production process, with white indicating newly captured pictures.

Daniel, a colleague of Peter, was hired as a driver, and he and Richie got busy mapping out the city, while I did further development of the production tools and web site. Randy soon came and helped out.

Below: The MapJack team, from left to right: Randy, Bjorn, Richie, Peter and Daniel. We featured this Easter egg photo in our San Francisco MapJack web site.

Peter remained in Sweden writing business documents, and eventually came to visit us in Daily City. It became clear to us that it was a mismatch of personalities, so we had to continue without Peter, a sad day for everyone involved. 

Mid May we had mapped all of San Francisco, a huge feat of 250,000 locations, or 1,000,000 pictures. We were in our final stage of publishing all content.

On May 29, we visited Where 2.0, a conference for the mapping industry. Google were one of the main exhibitors, and we were excited to speak with them about MapJack. Optimally we could sell the company to them and make some good money. I also remember a small company called Twitter having an exhibition booth there, but I just didn't understand why their "Internet SMS messages" would ever become popular...

Google had an exciting announcement to make, and we eagerly took seat in the auditorium. As their presentation unfolded I watched in surreal disbelief about Google Street View, their exact copy of my idea. It was even hard to take it all in, it seemed so strange. Their solution was slick. Instead of four separate images from each location, they had a seamless 360 degree panorama. And their user interface seemed more polished. But the image quality was lacking.

After the presentation we had a chat with them. Their tech-guy could see my sad look, and said "Don't worry, I've been in many start-ups that failed, this time I was lucky to sell it to Google, don't give up!"

We all took a long break to think about what to do next. I remember driving around the Bay area, depressed and confused. It felt like one year of hard work down the drain.

New technology, new business model

Next Randy tried hard to sell the company to others in the mapping business; Apple, Microsoft, and the likes. But no success. 

I went back to Thailand, and I felt we deserved a second chance. I worked hard to implement a panorama stitching solution, and reworked the web site interface to use Adobe Flash to show the panoramas with a 3D feel. I produced more during June than the previous three months combined, and those months had been the busiest in my life. Especially the panorama viewer was really hard to do. Lots of math and 3D optimizations involved.

Now we had some promise to succeed, because our images were much clearer than Google's, and our panorama viewer more fluid and slick, setting the industry standard for panorama viewers at the time. And personally I favored our way of indicating panorama locations as blue dots, over Google's way with lines.

Below: MapJack panoramas to the left, Google Street View panoramas to the right. At this time Google was using video cameras, while we were using DSLRs.

We rebooted the web site, and used it as a showcase to try and get clients to map panoramas for. The idea was to find businesses that featured maps on their web sites, and sell panorama pictures to them. The ideal clients were yellow pages.

Our web site caught on, and quite a few reviewers at online magazines favored our web site to Google's, writing things like "MapJack beats Google Maps", "MapJack is Streetview with better pictures", "MapJack - A small sample of what Google will inevitably do". Later we also got listed on Time Magazine's "50 best websites of 2008".

Meanwhile Randy put more money into the business to get Richie and Brad map cities around the Bay area: San Jose, Palo Alto, Pacifica, Sausalito, Oakland. Later in July also Lake Tahoe and Yosemite.

In August Randy went over to present MapJack to a room full of Google executives in Mountain View, California. They were all very impressed with us, and wanted to hire us to work for them.  We both hesitated though. Randy because of his other commitments, me because I had a hard time seeing myself working for a big corporation, like I did for WM-data. It had made me burn out twice, so I felt that I didn't have it in me to fit in such an environment. Plus it meant that we had to close down MapJack and all the work and money that went into it. Google offered to compensate us for this, but still we declined. Sometimes I think back of this as a mistake. It could have been interesting to see the Google culture for a few years, and it might have opened a lot of doors.

On the horizon was also the competitor EveryScape, who launched in October and had similar technology, but focused on building their own service and selling ad space.

Prior to MapJack and Google Street View, there was really no street view as we know it today, just a few very simple attempts at similar things, and at a very small scale.

Joint venture in Phoenix

We got the attention of John Vatistas, owner of The Equitable real estate company in Phoenix, Arizona. I already thought that our technology would be a perfect fit for real estate, so it was exciting. Imagine the convenience of talking a virtual walk around the neighborhood of an interesting house for sale.

After a few meetings with John and his colleagues, I found the personal chemistry very much lacking. John was a loud mouth cowboy, and my instinct told me he was not trustworthy. But he seemed to be a sharp businessman, and we had no other business prospects at the time. John had the impression that we were alone with street view technology, which clearly was not the case, as anyone with five minutes on Google could have found out. So we had a hard decision to make, to leave this business or to take it on very shaky ground.

In the end we created a joint venture with him and set it up in Phoenix. We were to map Phoenix and take the technology to the next level. In September Richie, Brad and I moved in to a small house on West Viginia Ave. I got myself a Roland Vdrums digital drum kit to have something fun to do and take my mind off the frustrating business environment. While I got busy getting into meetings with hosting providers, business people and hiring web development staff, Brad and Richie started mapping the city.

A month later I could sense that John had cold feet. He seemed to not be willing to commit to our plans, and he was hard to get hold of. We had a contract with him where we would receive payments in certain installments. When the second installment was over due, he called to a meeting where he said that he was changing the terms of the contract. No more money to us, but he would finance the cost of making a pitch to a big real estate group, to take the company to the next level. He was pissed off because he felt that we had lied about street view competitors, when he had asked us a direct question about it. Technically we had not lied, but we did omit this fact from him, which made me incredibly uncomfortable at the time. But I also reasoned that he had the duty to do at least a little research about the business he went into, before signing any contracts. Plus I felt that the fact that our solution was so superior to Google's and EveryScape's, would make this a non-issue. 

It felt like it was a joint venture made for doom, the minute we signed it. It was depressing that I had endured so much time in this hostile environment for nothing. I was pissed off too, and intended to force him to live up to the contract. A lawyer confirmed that we had a solid case, but that it would take years, and be very stressful to litigate.

John's business associate Clay Baldwin, pitched in around 10% of the total contract money, to try and find a way forward. I took his check, cashed it the very same day, and immediately left the country for Thailand, where I emailed everyone involved that I wouldn't lift a finger until all of the contract money was in escrow. This of course enraged the people in Phoenix. After we wrapped, Brad lingered behind and continued to map Phoenix for Clay. A year later Randy received a threat of litigation from John and we transferred back the money. 

Mapping Thailand

The fall and winter of 2007 I remained in Thailand and continued development to try and capture spherical panoramas, and general improvements in reliability. A standard DSLR camera isn't really made for this industrial level capture. They wear out, but usually last for 150,000 shots. But the main problem was that they skipped shots, so there would be a few random pictures missing from each camera. 

In December 2007 Randy and I went to Abu Dhabi, UAE, to discuss a deal with C4, a subsidiary of Emirates Advanced Investments. They wanted us to map all of Abu Dhabi on their behalf, and that we should censor out all parts of pictures with women not dressed moderately (in skirts, etc). They were all very professional, and spoke good English, but the deal unfortunately fell through.

Below: In the lounge of the Emirates Palace, preparing for the business meetings.

During the winter Brad and Richie came to stay in the house next to mine in Chiang Mai, and I held training sessions with them so they could be self reliant on making and fixing all equipment. In February 2008 they mapped Chiang Mai, using my Toyota Vios car. 

In March 2008 we sold a camera unit and production software to Wontech Vision, to use for mapping in Mexico.

Richie and I spent the spring of 2008 on a tour to map Ayutthaya, Pattaya, Krabi and Phuket, where we were joined by Randy to chill and dive in the ocean for a while. We were slowly building our panorama database, while looking for clients. Over the course of time I had also become a very close friend with Randy, who always stood up for me, and tried to shield me from frustrating details in the business.

Above: Richie and me in Ayutthaya, now using a custom made mount. Below: Randy and girlfriend Ailing, spreading the word on MapJack at conferences.

The mapping of Pattaya created a big controversy. We deliberately mapped the alleys where the hookers hung out, to stir up some interest, and some foreigners got caught in the process. But in our defense we didn't capture anything that a tourist couldn't do. 

The Sydney Morning Herald reported:

During the summer of 2008 we landed a deal with Swedish yellow pages service Hitta.se, to do a pilot of Stockholm, which was shot in August. Another deal with CanPages, a Canadian counter part, got us Vancouver, Squamish and Whistler in November 2008.

By the fall of 2008 I had enough of MapJack and resigned. It had been long over due. I had worked myself into the ground, and it was not fun anymore.

Privacy concerns over street view technology were growing in the US and Canada. The Canadian government set up a commission to investigate the issue, and spokespersons from CanPages and Google were summoned. Luckily for MapJack, the CanPages representative was very eloquent, and we had already developed a solid procedure for privacy concerns, where we would remove any image reported to us within 24 hours. Google appeared almost dysfunctional in this regard, which was a small victory for us.

New CEO and cooperation with SimplifyLogic

My departure from MapJack created a vacuum that Randy tried to fill. An old neighbor in Hong Kong seemed to be able to turn the business around. Stewart Bowers, an American, took charge as the new CEO in October of 2008, and brought in his old friend Alex Bracewell to take care of the technology. Alex owned software development company SimplifyLogic in Manila, Philippines, with around 15 people on staff. 

I didn't care at first, because it didn't affect me, I was already out of the company. But Stewart pushed me hard to come back and do some work. This lead to the second wave of MapJack. We did a technology transfer to Manila so they would run all production and future development. I remained as technical support, which became very frustrating, because it soon turned out that the Manila team were sloppy and this created a lot of grief for us.

Somehow we managed to map Toronto and Montreal for CanPages and 30 of the major cities of Sweden for Hitta.se during 2009. In 2010 we did Latvia and Puerto Rico by help from new employee Roberto Uzcategui. The Venetian in Macau was also done. I was fairly passive during these years, giving Stewart and Randy free reign over the company.

SimplifyLogic rewrote the software on a Linux platform, redesigned the MapJack web site, and created a player for mobile devices, in an effort to generate interest. They also collaborated with producers of gigapixel panoramas. At one point the company was 48 heads strong. I could see some good effort from Stewart to secure funding and make new deals, but my growing discontent with SimplifyLogic made me try to regain more control, so I set up a branch in Chiang Mai. I wanted peace of mind. Randy and I also suspected that Alex had been overcharging for the production, so I wanted to prove to them it could be done cheaper.

Chiang Mai branch

In November of 2011 I rented a three story office building in the center of Chiang Mai, and hired an office manager and two programmers.

We invited students from the nearby university to come and get training and work for us, and it was well received. Tim Martin, a photographer Randy met in Hong Kong, came and joined forces to drive the mapping car, and we re-mapped Chiang Mai in higher picture quality. We also experimented with mapping indoor locations such as restaurants and the local zoo. 

Below: The Chiang Mai staff (standing) and the students (seated).

We were doing fairly well in Chiang Mai, but things quickly deteriorated in Manila. Inevitably MapJack fell apart in February 2012. This left the company in debt to SimplifyLogic, which used this leverage to transfer ownership of all intellectual property (the MapJack name, all source code and engineering drawings) to them. In effect this was a hostile takeover of the company.

The intellectual property dispute remains unsolved, so its unclear who owns MapJack. Randy keeps mapjack.com running, and sometime someone shows interest, but with the massive roll-out of Google Street View, it has pretty much killed any competition. Adobe Flash is no longer supported in modern browsers, so the web site does not work anymore.

Mapio

In December 2012, Swedish yellow pages service Eniro wanted us to map all of Sweden. It seemed like a done deal so we rebooted MapJack as Mapio. Randy put in the money and formation documents, while I rented an office space in Chiang Mai, brought in equipment and hired five staff and was on standby for the project. I also re-wrote the entire production software to get around the intellectual property issues, and to make it more efficient and with more quality control focus.

Below: The Mapio office entrance, and our staff.

However the deal fell through at the last minute, and that really broke my spirit in a bad way.

We decided to send Tim to map Iceland and Bali, in a last effort to generate some interest, but we had close down the Mapio business in June 2013.

Conclusion

The adventure lasted from 2006 to 2015, and it was an interesting time. I feel that 90% of the exciting stuff happened within the first two years, and I wish I had moved on to something else after that. MapJack really dragged me down for a number of years. But overall it is something fun and valuable to look back on. A lot of lessons learned.

Below follows some selected images from MapJack, as they were shown on the web site.