Geolocating your holidays

Hi there

Tinkering with Python once again... Just for the sake of it :)

The idea is to generate a web page containing a Google Map with marks of your photo timestamps, geolocated in the map. I'm pretty sure Google Photos is able to do something similar (and better), but where's the fun in that?

For instance, I spent a few days in Kauai this summer, and this is the resulting map:

{% img center /img/20-12-2016-geotagging_map.gif 'Kauai photos timestamp' %}

Each mark is a photo I took, and you can see its timestamp moving the mouse over the marker...

Nothing fancy, but in case it is useful to you, this is the code:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
#!/usr/bin/env python
"""
Giving a directory as input:
1) the script will extract the dates and GPS coordinates of the NEF files
2) will add a marker in a Map 
3) print the HTML file

Dependencies: 
- EXIF data has to contain GPS tags
- exifread module installed
- Google Map API credentials
"""
import exifread,sys,glob

G_API_KEY = '<GOOGLE_MAP_API_TOKEN>'

html="""<!DOCTYPE html>
<html>
<head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
    <meta charset="utf-8">
    <title>Complex icons</title>
    <style>
      html, body {
        height: 100%;
        margin: 0;
        padding: 0;
      }
      #map {
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="map"></div>
    <script>

function initMap() {
  var map = new google.maps.Map(document.getElementById('map'), {
    zoom: 10,
    center: {lat: xxxLATxxx, lng: xxxLONGxxx}
  });

  setMarkers(map);
}  

xxxCAPTURESxxx

function setMarkers(map) {
  for (var i = 0; i < captures.length; i++) {
    var capture = captures[i];
    var image = 'data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20width%3D%2238%22%20height%3D%2238%22%20viewBox%3D%220%200%2038%2038%22%3E%3Cpath%20fill%3D%22%23808080%22%20stroke%3D%22%23ccc%22%20stroke-width%3D%22.5%22%20d%3D%22M34.305%2016.234c0%208.83-15.148%2019.158-15.148%2019.158S3.507%2025.065%203.507%2016.1c0-8.505%206.894-14.304%2015.4-14.304%208.504%200%2015.398%205.933%2015.398%2014.438z%22%2F%3E%3Ctext%20transform%3D%22translate%2819%2018.5%29%22%20fill%3D%22%23fff%22%20style%3D%22font-family%3A%20Arial%2C%20sans-serif%3Bfont-weight%3Abold%3Btext-align%3Acenter%3B%22%20font-size%3D%2212%22%20text-anchor%3D%22middle%22%3E' + capture[3] + '%3C%2Ftext%3E%3C%2Fsvg%3E';
    var marker = new google.maps.Marker({
      position: { lat: capture[1], lng: capture[2] },
      map: map,
      icon: image,
      title: capture[0],
      zIndex: capture[3]
    });
  }
}
    </script>
    <script async defer
        src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&signed_in=true&callback=initMap"></script>
  </body>
</html> """ 

def formatme(mydata,reference):
    coordenates=(str(mydata).replace("[","").replace("]","").replace(" ","").split(","))

    d=coordenates[0]
    m=coordenates[1]
    s=coordenates[2]

    ref = str(reference).strip()

    d = int(d.split("/")[0]) / int(d.split("/")[1]) if "/" in d else int(d)
    m = int(m.split("/")[0]) / int(m.split("/")[1]) if "/" in m else int(m)
    s = int(s.split("/")[0]) / int(s.split("/")[1]) if "/" in s else int(s)

    coor = d + (m / 60.0) + (s / 3600.0)
    return coor if ref == "N" or ref == "E" else -coor

def find_pics(pics_path):
    #Some more logic could be added here
    files = glob.glob(pics_path + "/*.NEF") 
    files.sort(key=lambda x: os.path.getmtime(x))
    return files

def get_GPS_data(id,path):
    try:
        #Get the EXIF tags
        with open(path, 'rb') as f:
            tags = exifread.process_file(f,details=False)
    except Exception as e:
        data = ["skip"]

    #Check we got the GPS tag data
    if "GPS GPSLongitude" not in tags and "GPS GPSLongitudeRef" not in tags:
        data = ["skip"]
    else:
        data = [path,"{0}".format(tags["EXIF DateTimeOriginal"]),formatme(tags["GPS GPSLatitude"], \
                tags["GPS GPSLatitudeRef"]),formatme(tags["GPS GPSLongitude"], tags["GPS GPSLongitudeRef"]),id]

    return data

if __name__ == "__main__":
    try:
        path=sys.argv[1]
    except:
        print("Usage: " + sys.argv[0] + " <path>")
        sys.exit(1)

    #List of pictures to process
    pics_list=find_pics(path)

    #List of pics, dates, lat, long, id
    captures = []
    for _, pic in enumerate(pics_list):
        item = get_GPS_data(_, pic) 
        if "skip" not in item: captures.append(item)

    # Create the JS code with the list of coordenates
    html_loop = "var captures = ["
    for capture in captures:
        html_loop = html_loop + "".join("['" + str(capture[1]) + "'," + str(capture[2]) + "," + str(capture[3]) + \
                    "," + str(capture[4]) + "],\n")
    html_loop = html_loop[:-2] + "]; "

    #Update center LAT/LONG  coordenates
    html = html.replace("xxxLATxxx",str(captures[0][2]))
    html = html.replace("xxxLONGxxx",str(captures[0][3]))
    #Update list of points
    html = html.replace("xxxCAPTURESxxx",html_loop)
    #Update API KEY
    html = html.replace("YOUR_API_KEY",G_API_KEY)

    print(html)

Stored in Github

The script is so simple that I am not even opening a file to write the output... Feel free to do it yourself, or redirect the output to a file:

# python extract_geolocation_batch_to_html.py > mymap.htm

Take care out there...

\\psgonza

Spam via Google Analytics (update)

I am not saying Russia has anything to do with the US presidential elections results, but checking Google Analytics, the referral spam is kind of "funny"...

{% img center /img/16122016_spam.JPG 'spam' %}

And the location:

{% img center /img/16122016_spam2.JPG 'spam' %}

Apparently a lot people followed their advice and voted for Trump.

Related

Later

Trying to minimize tracking in mobile platforms

Random thoughts about how to reduce your "big data footprint"

1) Social networks

I have managed to reduce my "wasted" time in social networks to the bare minimum...

  • Earlier this year I "quit" G+ (it's a pitty, it had a lot of potential, but it ended up badly)
  • I uninstalled FB app from my smartphone months ago, so I use the browser to log in FB (I keep Twitter app installed though)
  • I spend just a few minutes in FB/Twitter every day. Pretty low profile for today's standards, I guess.

But the tracking ads in FB are driving me crazy.

Quick examples:

a) I was having a discussion with some friends about portable espresso machines (yes, they do exist), so I googled it and went on with my life. Next time I logged in FB, there it was... Portable espresso machine:

{% img center /img/photo_2016-12-14_11-38-13.jpg 'ads' %}

b) Another discussion about smart trackers, another search and 15 minutes later I am seeing promoted posts in Facebook.

Probably I was using Chrome Browser and Google Search for this, so not only FB is to blame... And don't get me wrong, I don't mind Mark making loads of money sharing (should I say "selling"?) people's info (life?), but just not mine...

As I said, I already removed Facebook Android application from my phone (and you should probably do the same) and I was using my main browser (Chrome) to log in, but third-party cookies are not helping here... I have decided to use Firefox browser mainly for Facebook. It was already installed in my smartphone, so I simply created a shortcut to the FB url in the "desktop"

If you are not a heavy FB user, this will do the job... It is better than the FB lite application, and you will be able to isolate your browsing history from your FB session (I'm sorry Mark)

2) Google ads + location

It's not easy to stay away from Google when you are using Android devices... I am a happy CyanogenMod user (both in my OnePlus One and my old Samsung S3) but Google services are the core of the OS, and, truth being told, they tend to make your life easier. It comes with a price, of course: "If You're Not Paying for It, You're the Product"

There are a lot of information in the Internet about Google services alternatives such as leavegooglebehind.com but it is not an easy task (see Bryan's post series in Networkworld). I haven't gone that far yet but there are some easy alternatives to Google apps that do the job for me:

  • Using different internet browsers such as Firefox (not "instead of" but together with Chrome)
  • Duckduckgo as primary search engine (instead of Google Search)
  • Telegram as IM client (instead of Hangouts/Allo)
  • Piktures as gallery app (instead of Photos)
  • Dropbox as shared storage (instead of Drive)

Some other applications such as GMail, GMaps, Youtube, GDocs, etc are not so easy to replace, and I wouldn't look for an alternative unless you are really decided to live out of Google's kingdom.

And one step further: disable GPS, disable WLAN when not connected, remove old SSIDs from your WLAN, disable Google history, and location services... There are many things to do if you are concerned about your privacy. However, being connected 24/7 means there will be a track of your activities/locations somewhere, like it or not.

Trying to keep the same level of privacy as in 1990's, when people were unable to reach you if you weren't at home, is kind of impossible nowadays. Embrace it or go find a nice cave because you are going to have a hard time in the next 5-10 years.

Let's be careful out there

//psgonza

PS: And rest assured that not only companies like Google, FB, Apple, Amazon, etc are willing to track you... Bad guys and goverments have a lot of interest in your digital life as well...

Spam via Google Analytics

I have to give it to them... People are pretty damn creative... Email spam, Whatsapp spam, and why not... Referral spam!

{% img center /img/spam_via_analytics.JPG 'spam' %}

People should put all this effort and time in making the world a better place

Minimal Cygwin Python data science installation

A few days ago a made a fresh installation of Cygwin64 in order to tinker with Pandas (just for fun) but instead of installing Anaconda, I decided to install each component manually... I will write it here because I'm sure I will need it in the future (and it might be handy for someone else).

First things first, package management in Cygwin is a bit of a pain without apt-cyg... So download it from here, save it as "apt-cyg" in your $HOME and install it:

install apt-cyg /bin/