Geek as a platform

SFTP oneliner (as SCP)

Quick note to self:

Secure Copy (scp) (a one line command very easy to include in bash scripts) can be replaced by a similar Secure File Transfer (sftp) command, but the documentation is not very clear...

For instance, this command to transfer a file to a remote host using SCP:

scp src_file user@remote_host:/remote/path/dst_file

Can be replaced by this one, using SFTP:

sftp user@remote_host:/remote/path/ <<< $'put src_file dst_file'

Example:

# sftp root@192.168.1.100:/tmp/ <<< $'put /tmp/test test_sftp'
Connected to 192.168.1.100.
Changing to: /tmp/
sftp> put /tmp/test test_sftp
Uploading /tmp/test to /tmp/test_sftp
/tmp/test                                          100%    0     0.0KB/s   00:00
#

The idea is taken from a few StackOverflow threads, all the credit to them... I decided to put it together here, so I can find it easily.

Later

2017 resolutions

'2017resolutions'

Another entry in my personal online notebook (considering the number of visits of this site, I don't think it should be called blog or website)

We are only 11 day into the new year, so it is not too late for my 2017 resolutions. I'd rather post them here than keep them in a piece of paper on my desktop because, apparently, making goals "public" helps to succeed (or I can be publicly shamed at the end of the year)

Here they go:

1) The classics:

  • Work out more often: At least 3 times per week. No matter if it rains or snows.
  • Eat (more) healthy: It seems I'll be stuck in Spain for a few months (at least until April), so it will be a good opportunity to prepare some green juices (I bought a blender a couple of years ago, and I think I've turned it on 2 times).
  • Read (more) books: Goodreads helps me to track my reading habits... And they are far from impressive, around 12 books per year. It doesn't sound too bad, but considering I don't have family responsibilities, it is not enough. I think I should blame Netflix for this :)
  • Study something new: There are countless online courses nowadays available for free. I've already done a couple of them in edX and I'd like to do more. Coursera has some really interesting courses as well.
  • Improve my English pronunciation: My English is quite good, but still sounds like a spaniard speaking English.

2) Job related:

  • Look for a different role/position: I've been doing pretty much the same stuff for way too long. Time to change. Period.
  • Ramp up my python skills: I am kind of stuck with Python. It is enough for my current role (system engineering in a big telco, not development) but I'd say it is not good enough if I want to do something different (more IT related).
  • Learn some new languages: Go?
  • Ansible: I've been playing with Ansible every now and then for a couple of years, creating some playbooks for myself, but nothing fancy. I don't have the opportunity to use it in my current gig, so I'd need to look for a way to include it in my daily work.

3) Hobby related:

  • Fix my home network: 1 ISP router + 1 AP router + 1 openWRT (as firewall) + 1 NAS + several PC/laptops/smartphones/raspberry pi's = a hell of a mess.
  • Soldering: I have a bunch of ESP8266 chips laying around in a box. I should put them to good use.
  • Photo library housekeeping: I have traveled a lot these past years, and I have tons of photos in my NAS that I have not even seen. It needs a cleanup.
  • Post more (and better): I registered this domain back in 2005... And, even though no one reads this (the number of daily page reads is close to zero), I like to post stuff here. It's a good way to keep things available (thanks to github of course). I'd like to post more stuff, and if possible, more interesting.

I'd say this should be enough for 2017.

\\psgonza

My 2016 in photos

(Pretty big pictures, sorry about that)

Saying hello to another year with some pictures... 2016 is been a good year, both personally and professionally:

  • I've been working in Japan for several months again... Best place for living as expat (IMO)

'Japan'

  • April: I never really liked south east Asia that much (too hot for my taste), but I have to say I had a really good time in Vietnam and Cambodia... Looking forward to going back there soon!

'vietnam_cambodia'

  • July: Raising sun at the top of Mt Fuji

'fuji'

  • August: Hawaii is the place to be. Period.

'hawaii'

  • December: Vienna (European capital of Xmas) & Salzburg

'austria'

I hope 2017 is as good as 2016, although I'd like to make some changes...

Happy New Year everyone

\\psgonza

Related: 2015 in photos, 2014 in photos

My books in 2016

We just said goodbye to 2016, and as in 2015, here goes my Goodreads summary:

'goodreads'

Unfortunately, I failed my reading challenge... I read 10 of the 13 books in the first half of the year, but I have been stuck with a Python book for months. And podcasts have not helped much :)

Happy reading!

\\psgonza

Geolocating your holidays

Hi there

Tinkering with Python once again... Just for the sake of it :)

The idea is to generate a web page containing a Google Map with marks of your photo timestamps, geolocated in the map. I'm pretty sure Google Photos is able to do something similar (and better), but where's the fun in that?

For instance, I spent a few days in Kauai this summer, and this is the resulting map:

'Kauai photos timestamp'

Each mark is a photo I took, and you can see its timestamp moving the mouse over the marker...

Nothing fancy, but in case it is useful to you, this is the code:

#!/usr/bin/env python
"""
Giving a directory as input:
1) the script will extract the dates and GPS coordinates of the NEF files
2) will add a marker in a Map 
3) print the HTML file

Dependencies: 
- EXIF data has to contain GPS tags
- exifread module installed
- Google Map API credentials
"""
import exifread,sys,glob

G_API_KEY = '<GOOGLE_MAP_API_TOKEN>'

html="""<!DOCTYPE html>
<html>
<head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
    <meta charset="utf-8">
    <title>Complex icons</title>
    <style>
      html, body {
        height: 100%;
        margin: 0;
        padding: 0;
      }
      #map {
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="map"></div>
    <script>

function initMap() {
  var map = new google.maps.Map(document.getElementById('map'), {
    zoom: 10,
    center: {lat: xxxLATxxx, lng: xxxLONGxxx}
  });

  setMarkers(map);
}  

xxxCAPTURESxxx

function setMarkers(map) {
  for (var i = 0; i < captures.length; i++) {
    var capture = captures[i];
    var image = 'data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20width%3D%2238%22%20height%3D%2238%22%20viewBox%3D%220%200%2038%2038%22%3E%3Cpath%20fill%3D%22%23808080%22%20stroke%3D%22%23ccc%22%20stroke-width%3D%22.5%22%20d%3D%22M34.305%2016.234c0%208.83-15.148%2019.158-15.148%2019.158S3.507%2025.065%203.507%2016.1c0-8.505%206.894-14.304%2015.4-14.304%208.504%200%2015.398%205.933%2015.398%2014.438z%22%2F%3E%3Ctext%20transform%3D%22translate%2819%2018.5%29%22%20fill%3D%22%23fff%22%20style%3D%22font-family%3A%20Arial%2C%20sans-serif%3Bfont-weight%3Abold%3Btext-align%3Acenter%3B%22%20font-size%3D%2212%22%20text-anchor%3D%22middle%22%3E' + capture[3] + '%3C%2Ftext%3E%3C%2Fsvg%3E';
    var marker = new google.maps.Marker({
      position: { lat: capture[1], lng: capture[2] },
      map: map,
      icon: image,
      title: capture[0],
      zIndex: capture[3]
    });
  }
}
    </script>
    <script async defer
        src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&signed_in=true&callback=initMap"></script>
  </body>
</html> """ 

def formatme(mydata,reference):
    coordenates=(str(mydata).replace("[","").replace("]","").replace(" ","").split(","))

    d=coordenates[0]
    m=coordenates[1]
    s=coordenates[2]

    ref = str(reference).strip()

    d = int(d.split("/")[0]) / int(d.split("/")[1]) if "/" in d else int(d)
    m = int(m.split("/")[0]) / int(m.split("/")[1]) if "/" in m else int(m)
    s = int(s.split("/")[0]) / int(s.split("/")[1]) if "/" in s else int(s)

    coor = d + (m / 60.0) + (s / 3600.0)
    return coor if ref == "N" or ref == "E" else -coor

def find_pics(pics_path):
    #Some more logic could be added here
    files = glob.glob(pics_path + "/*.NEF") 
    files.sort(key=lambda x: os.path.getmtime(x))
    return files

def get_GPS_data(id,path):
    try:
        #Get the EXIF tags
        with open(path, 'rb') as f:
            tags = exifread.process_file(f,details=False)
    except Exception as e:
        data = ["skip"]

    #Check we got the GPS tag data
    if "GPS GPSLongitude" not in tags and "GPS GPSLongitudeRef" not in tags:
        data = ["skip"]
    else:
        data = [path,"{0}".format(tags["EXIF DateTimeOriginal"]),formatme(tags["GPS GPSLatitude"], \
                tags["GPS GPSLatitudeRef"]),formatme(tags["GPS GPSLongitude"], tags["GPS GPSLongitudeRef"]),id]

    return data

if __name__ == "__main__":
    try:
        path=sys.argv[1]
    except:
        print("Usage: " + sys.argv[0] + " <path>")
        sys.exit(1)

    #List of pictures to process
    pics_list=find_pics(path)

    #List of pics, dates, lat, long, id
    captures = []
    for _, pic in enumerate(pics_list):
        item = get_GPS_data(_, pic) 
        if "skip" not in item: captures.append(item)

    # Create the JS code with the list of coordenates
    html_loop = "var captures = ["
    for capture in captures:
        html_loop = html_loop + "".join("['" + str(capture[1]) + "'," + str(capture[2]) + "," + str(capture[3]) + \
                    "," + str(capture[4]) + "],\n")
    html_loop = html_loop[:-2] + "]; "

    #Update center LAT/LONG  coordenates
    html = html.replace("xxxLATxxx",str(captures[0][2]))
    html = html.replace("xxxLONGxxx",str(captures[0][3]))
    #Update list of points
    html = html.replace("xxxCAPTURESxxx",html_loop)
    #Update API KEY
    html = html.replace("YOUR_API_KEY",G_API_KEY)

    print(html)

Stored in Github

The script is so simple that I am not even opening a file to write the output... Feel free to do it yourself, or redirect the output to a file:

# python extract_geolocation_batch_to_html.py > mymap.htm

Take care out there...

\\psgonza