bynario.com

NEF batch processing in MacOS (from CLI)

psgonza — Tue, 15 May 2018 00:05:15 +0200

Hi there

In case someone has to scratch the same itch as me (converting and resizing big NEF files to ease the visualization over the network), this is (by far, I'd say) the easiest way to generate (and resize) a jpg file from a big fat Nikon RAW file (NEF):

psgonza/Nikon$ for i in `ls -1 *.NEF`; 
do 
sips -s format jpeg -s formatOptions 50 -Z 3000 "${i}" --out "jpg/${i%NEF}jpg"; 
done

Above command will:

Go through all the NEF files in current directory
Generate a jpg from the original nef file
Set jpg quality at 50%
Resize the image to max 3000px (either width or height, whatever is bigger)

It took A WHILE to process all the files (in my case, 1003 pictures), but I'd say it is worthy... I ended up dealing with (aprox) 1MB jpg files, instead of the original 25MB NEF files I had.

I am fairly new to the Mac OS world so I didn't know about sips until 30 min ago, although it has been included by default in Mac OS since 2003...

Pretty handy

PSA: MiniKeePass and key files

psgonza — Sat, 17 Feb 2018 00:01:17 +0200

In case you plan to use MiniKeePass on your iphone with key files (which I guess is highly recommended if you plan to share your kdb via Dropbox/Drive/you_name_it), pay special attention to the last line of their help page:

Using Key Files

MiniKeePass can open KeePass 1.x/2.x files that use a key file instead of or in
addition to a password.

Steps:

Load your KeePass database and key file in MiniKeePass using iTunes, Dropbox, etc.
Open your KeePass file in MiniKeePass
When prompted for your password you can enter a password and/or select a key file

Note: MiniKeePass will automatically select your key file if it has the same
filename as your KeePass file but with a .key extension.

Basically, you have to upload your data base and the key file with the same name (ie: MyDB.kdb and MyDB.key) to your_cloud_storage_service_here, and then, open both of them with MiniKeePass. Otherwise, all you will see in the Key-file screen is "None".

I learnt it the hard way and I lost 15 minutes of my life trying to import a key file with a different name than the KeePass database.

The devil is in the detail, as usual.... ;)

///psgonza

KeePass is great, you should totally use. MiniKeePass is just fine, does the job too.

Bye bye 2017, hello 2018

psgonza — Sun, 31 Dec 2017 23:59:59 +0200

We have a coouple of hours left from this 2017... Or maybe you are already in 2018, or half a day away if you are in some places of the US...

Anyways, happy new year!

There will be time to analyze what was good, bad or needs to be improved in the coming days... Meanwhile, enjoy New Years Eve!

\\psgonza

String manipulation exercise: Perl, Python, Awk

psgonza — Fri, 25 Aug 2017 08:25:00 +0200

Here comes a small comparation of the performance of Perl, Awk and Python while parsing and splitting lines in a BIG ldif file with thousands or millions of subscriber profiles like this one (I created this ldif file as an example, just for the sake of clarity)

As in the example, one of the attributes in my ldif files was a huge base64 string (more than 10k bytes long) in a single line, which is not supported by slapadd/slapd (I have checked LDIF rfc and I don't see any mention to the 4096 bytes limitation, so it could be our own implementation, not sure about this), so the idea was to spit this line into 76 characters lines (as per recommendation)... Something like this:

Original line (short version):

...
Service: SSBhbSBoYXBweSB0byBqb2luIHdpdGggeW91IHRvZGF5IGluIHdoYXQgd2lsbCBnbyBkb3duIGluIGhpc3RvcnkgYXMgdGhlIGdyZWF0ZXN0IGRlbW9uc3RyYXRpIGV2ZXJ5IHN0YXRlIGZa
...

Replaced by:

...
Service: SSBhbSBoYXBweSB0byBqb2luIHdpdGggeW91IHRvZGF5IGluIHdoYXQgd2lsbCBnbyBkb3
 duIGluIGhpc3RvcnkgYXMgdGhlIGdyZWF0ZXN0IGRlbW9uc3RyYXRpIGV2ZXJ5IHN0YXRlIGZa
...

(Notice the blank at the beginning of the second line)

My first approach was to use Python standard module textwrap:

import sys
import textwrap

try:
    ldiffile = sys.argv[1]
except:
    print("Input file missing... Exit")
    sys.exit(0)


with open(ldiffile, "rU") as f:
    for line in f:
        if line.startswith("Service:"):
            print(textwrap.fill(line, width=76,subsequent_indent=' '))
        else:
            print(line.strip())

It was really simple and produced the expected result, but it was ridiculously slow. RIDICULOUSLY. Really, difficult to believe...

So I decided to do something similar, but this time I dealt with the line split myself:

import sys

try:
    ldiffile = sys.argv[1]
except:
    print("Input file missing... Exit")
    sys.exit(0)

def fixlen(s, n):
    #First line, no blank at the beginning of the line
    print (s[:n])
    s = s[n:]
    #now, starting with blank
    while s:
        print (" " + s[:n])
        s = s[n:]

with open(ldiffile, "r") as f:
    for line in f:
        if line.startswith("Service:"):
            fixlen(line,76)
        else:
            print(line.strip())

The results I got were way better, but still, it took more than I expected... Time to give it a try with other tools: AWK and Perl

Basically I "translated" the python script to awk and perl, almost to the letter...

Perl version:

#!/usr/bin/perl
my $noident = 0;
while ($line = <>) {
  chomp($line);
  if ( $line =~ /^Service:.*/)
  {
    for (unpack("(A76)*",$line)) {
        if ($noindent == 0) {
            print "$_\n";
            $noindent = 1; }
        else { print " $_\n"; }
    }
  }
  else
  {
    $noindent = 0;
    print "$line\n";
  }
}

Awk version:

awk '
  BEGIN {
    indent = 0;
    i=0;
  }
  {
  if ( $0 ~ /^Service:.*/) {
    while(i<=length($0)){
        if ( $indent == 0 ) { printf "%s\n", substr($0,i,76);i+=76;indent=1; }
        else { printf " %s\n", substr($0,i,76);i+=76;}
   }
   }
   else {
        print $0;
   }
  }' $1

I "faked" several input files with thousands of lines (discarding the output), executed all the scripts in Cygwin64, and then checked the time it took.. Something like this:

time python3 textwrap.py XXXX.ldif &> /dev/null
time python3 pythonv1.py XXXX.ldif &> /dev/null
time python3 pythonv2.py XXXX.ldif &> /dev/null
time perl split.pl  XXXX.ldif &> /dev/null
time ./split.awk XXXX.ldif &> /dev/null

All of them generated the exact same output (except the textwrap version that handles the first line in a different way), but the execution time differs:

{% img center https://raw.githubusercontent.com/psgonza/bynario/master/results_table.JPG 'results_table' %}

It is easier to see in a chart:

{% img center https://raw.githubusercontent.com/psgonza/bynario/master/results_chart.JPG 'results_chart' %}

My takeaways after this small exercise:

Awk rocks.
Perl's black magic is almost as fast as awk.
Python is not really fast at file processing. Yes, there are ways to improve this, by splitting in chunks, parallel processing and whatnot... But it takes more than 15 lines.
Stay away of textwrap module for heavy usage.

It was fun... Bye!

PS: As you can see in the results, there is a "Python v2" column which produced better results... It is a modification of the original script where I used a comprehension list approach:

import sys

try:
    ldiffile = sys.argv[1]
except:
    print("Input file missing... Exit")
    sys.exit(0)

def fixlen(s, n):
    first = True
    tmp = (s[0+i:n+i] for i in range(0, len(s), n))
    for x in tmp:
        #First line, no blank at the beginning of the line
        if first:
            print(x)
            first=False
        else:
            #now, starting with blank
            print(" " + x)

with open(ldiffile, "r") as f:
    for line in f:
        if line.startswith("Service:"):
            fixlen(line,76)
        else:
            print(line.strip())

Issues running gpg in a container

psgonza — Mon, 05 Jun 2017 00:01:17 +0200

In case it helps...

I am giving a try to this docker componse example, and for some strange reason, docker was stuck in this block of code in the Dockerfile for one of the components:

# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
        && apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
        && wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
        && wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
        && export GNUPGHOME="$(mktemp -d)" \
        && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
        && gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
        && rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
        && chmod +x /usr/local/bin/gosu \
        && gosu nobody true

Specifically, in this line:

gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4

It is a really simple command... I could ping the keyserver, I checked the website and everything looked ok, but it wasn't working, neither in the host machine or inside a container... The response was the same: timeout while getting the keys

So... Here comes netstat to the rescue:

# netstat -natop | grep gpg
tcp        0      1 192.168.1.10:34340      104.236.209.43:11371    SYN_SENT    8653/gpg2keys_hkp    on (7,10/3/0)

SYN_SENT? So it was trying to stablish the connection, but there wasn't any response from the remote host... In that port

It turns out I recently upgraded my internet connection, and the new router allows you to customize the firewall security levels (you know, low, medium, high and paranoid... yeah, I am using the latter). I noticed that port 11371 was not defined in as a "known service", so I wasn't able to reach it from within my home network.

As soon as I allowed the connection to that port:

# gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
gpg: solicitando clave BF357DD4 de hkp servidor ha.pool.sks-keyservers.net
gpg: /root/.gnupg/trustdb.gpg: se ha creado base de datos de confianza
gpg: clave BF357DD4: clave pública "Tianon Gravi <tianon@tianon.xyz>" importada
gpg: no se encuentran claves absolutamente fiables
gpg: Cantidad total procesada: 1
gpg:               importadas: 1  (RSA: 1)

So make sure all your ip flows are open, even for such a small thing as this one...

Take care out there!

Using different sudo passwords for the same host group in Ansible

psgonza — Wed, 31 May 2017 00:01:17 +0200

I'll write this down here before I forget.

I have 2 VPSs that I manage with Ansible and recently I updated the root password for one of the servers, and my old playbook, which worked fine if all the hosts in the group have the same password, stopped working.

I tried some stuff that didn't work, until I found out that the way to solve this is described in this link:

Ansible - Become

Basically I added a connection variable called ansible_become_pass for each of my servers, and assigned them with the variable names inside of my Vault. Like this:

[vps]
foo.com ansible_user=foo ansible_ssh_private_key_file=/server/id_rsa_foo ansible_become_pass='{{ foo_sudo_pass }}'
bar.foo.com ansible_user=bar ansible_ssh_private_key_file=/server/id_rsa_bar ansible_become_pass='{{ bar_sudo_pass }}'

The variables foo_sudo_pass and bar_sudo_pass are the root passwords for each server and they are stored in my Vault file (called passwords.yml).

Those variables will be readed at the time the playbook is executed... So you could have something as simple as this:

$ cat upgradeAll.yml
---
- hosts: vps
  become: true
  tasks:
  - name: "apt-get update"
    apt:
      upgrade: dist
      update_cache: yes
      cache_valid_time: 7200

And run the playbook with the verbose option to get some more information with this command:

$ ansible-playbook upgradeAll.yml -e@/server/playbooks/passwords.yml --vault-password-file vault_pass.txt -v

Let's see an example:

$ ansible-playbook upgradeAll.yml -e@/server/playbooks/passwords.yml --vault-password-file vault_pass.txt  -v
PLAY [vps] 
*********************************************************************************

TASK [Gathering Facts] 
*********************************************************************************
ok: [foo.com]
ok: [bar.foo.com]

TASK [read vars] 
*********************************************************************************
ok: [foo.com] => {"ansible_facts": {"foo_sudo_pass": "pass1", "bar_sudo_pass": "pass2", "test_sudo_pass": "testpass"}, "changed": false}
ok: [bar.foo.com] => {"ansible_facts": {"foo_sudo_pass": "pass1", "bar_sudo_pass": "pass2", "test_sudo_pass": "testpass"}, "changed": false}

TASK [apt-get update] 
*********************************************************************************
changed: [foo.com] => {"changed": true, "msg": "Reading package lists.
[...]

PLAY RECAP 
*********************************************************************************
foo.com                : ok=3    changed=1    unreachable=0    failed=0
bar.foo.com            : ok=3    changed=1    unreachable=0    failed=0

$

It is interesting to notice how I added the extra variables in the command by using -e@/server/playbooks/passwords.yml, otherwise it won't work!

Later!

Interview question: chmod chmod?

psgonza — Tue, 09 May 2017 13:13:17 +0200

Not sure why, today I remembered about a question from a job interview I did probably 5 years ago... It was a pretty easy question, and my answer was pretty dumb.

Q: "Ok, imagine you remove execution permissions to chmod... How would you put them back???"

A: "quick and dirty, I'd copy the binary from the machine sitting next to it".

How about that? I remember it was very late in the evening (I was working in Tokyo at that time and they called me from Europe), but anyway, there was no excuse. I think they guy who interviewed me is still laughing :)

There are multiple ways to solve it. Using perl, python, go, C++, etc allows you to change the permissions of any file, including chmod... Let me show you how I'd do it in python:

#Current permissions: 755
root@94d9407f1002:/# # ls -l /bin/chmod
-rwxr-xr-x 1 root root 56112 Feb 18  2016 /bin/chmod

#Let's remove execution permission
root@94d9407f1002:/# chmod -x /bin/chmod

#New permissions: 644 
root@94d9407f1002:/# ls -l /bin/chmod
-rw-r--r-- 1 root root 56112 Feb 18  2016 /bin/chmod

#Checking execution is not allowed
root@94d9407f1002:/# chmod +x /bin/chmod
bash: /bin/chmod: Permission denied

#Here comes python to the rescue
root@94d9407f1002:/# python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.chmod("/bin/chmod",0o755)
>>> exit()

#Confirming it worked as expected
root@94d9407f1002:/# ls -l /bin/chmod
-rwxr-xr-x 1 root root 56112 Feb 18  2016 /bin/chmod

Not sure why this came to my mind today... But in case someone asks you this very same question, don't f$ck it up as I did. Just breathe normally, and think...

\\psgonza

All the images missing... Thanks Dropbox :(

psgonza — Tue, 09 May 2017 00:01:17 +0200

I bet nobody noticed but all the images in the blog were not here anymore... No pictures, no logo, etc. Ooops!

I was using dropbox as storage for the blog images, and a while ago they decided to change the policy about their public folders. Bummer!

I don't think Dropbox is to blame here though. They provide a really good service (for free!) and I guess they have to draw a line somewhere.

Anyway, I will be storing the images in the server from now on, so I am updating the posts and changing the image paths.

Bye

Simple Vigenere cipher in Python (and 3)

psgonza — Fri, 10 Mar 2017 13:13:17 +0200

See: - Part 1/3 - Part 2/3

Last part of my series about Vigenere cipher. (3 post in a row? I am proud of myself :-P)

In my previous posts I already showed how to use Vigenere square to encrypt/decrypt text, so this time I'll follow the algebraic method described in the Wikipedia:

{% img center https://bynario.com/img/vigenere.jpg 'vigenere' %}

I'll use the same input, same key, and same alphabets as in previous exercises:

mykey:  "WHITE" 
input_text:  "en un lugar dela mancha de cuyo nombre no quiero acordarme" 
Position:           00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 
Ref alphabet(M):    A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 
Key alphabet(K):    B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z  A

(In the reference alphabet,we have shifted the letters one position, as the author did in the book, so K alphabet starts in "B" not in "A"

Encryption:

Now we are going to use numbers instead of the square approach.

If you remember the first post, the foundation of this cipher is the tuple (letter,key):

INPUT: EN UN LUGAR DE LA MANCHA
KEY:   WH IT EWHIT EW HI TEWHIT
TUPLE: ('E', 'W'),('N', 'H'),('U', 'I'),('N', 'T'), ('L', 'E'),('U', 'W'),[...]

Let's get the positions of each element in the tuple in M[] and K[]:

1st tuple: 'E' is in position 4 in our reference alphabet(M). 'W' is in position 21 in the Key alphabet(K)
2nd tuple: 'N' is in position 13 in M[], 'H' is in position 6 in K[]
3rd tuple: 'U' is in position 20 in M[], 'I' is in position 7 in K[]

('4','21'),('13','6'),('20','7'),('13','18'),[...]

So putting this in the mathematical notation:

C[i] = (M[i]+K[i]) mod len(M)

C[0] = (4 + 21) % 26 = 25
C[1] = (13 + 6) % 26 = 19
C[2] = (20 + 7) % 26 = 1
[...]

So the letter "E" in position 4 in M[] will be replaced by the letter in position 25 in K[], which is "A". The same way, the letter "N" in position 13 in M[] will be replaced by the letter in position 19 in K[], which is "U", and so on...

E -> K[25] ->A
N -> K[19] ->U
U -> K[1] ->C
N -> K[5] ->G
L -> K[14] ->P 
U -> K[15] ->Q
G -> K[12] ->N
[...]

Decryption:

Decryption works pretty much the same way... The message is calculated this way:

M[i] = (C[i]-K[i]) mod len(M)

Let's check step by step.. This is our input data:

INPUT: AU CG PQNIK HA SI FEJJPT
KEY:   WH IT EWHIT EW HI TEWHIT
TUPLE: ('A', 'W'),('U', 'H'),('C', 'I'),('G', 'T'), ('P', 'E'),('Q', 'W'),('N', 'H'),('I', 'I'),('K', 'T'),('H', 'E'),[...]

We have to look for the positon of the each letter (of each tuple) in alphabet K[]:

1st tuple: Position of letter "A" and "W" in K[], 25 and 21.
2nd tuple: Position of letter "U" and "H" in K[], 19 and 6.
3rd tuple: C,I -> 1, 7

Now, coming back to the formula:

M[0] = (C[0]-K[0]) mod len(M) = (25 - 21) % 26 = 4 
M[1] = (C[1]-K[1]) mod len(M) = (19 - 6) % 26 = 13  
M[2] = (C[2]-K[2]) mod len(M) = (1 - 7) % 26 = 20 
M[3] = (C[3]-K[3]) mod len(M) = (5 - 18) % 26 = 13

And looking for those positions in our reference alphabet M[]:

A -> M[4] -> E 
U -> M[13] -> N 
C -> M[20] -> U 
G -> M[13] -> N 
P -> M[11] -> L 
Q -> M[20] -> U 
N -> M[6] -> G 
I -> M[0] -> A 
K -> M[17] -> R 
[...]

Let put this into python code:

#!/usr/bin/env python
import string

mykey="WHITE"
input_text="en un lugar de la mancha de cuyo nombre no quiero acordarme"
code_text="AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM"

# Alphabet used as reference (M)
# ABCDEFGHIJKLMNOPQRSTUVWXYZ
source = string.ascii_uppercase

# Key alphabet (K) shifted 1 position to the left
# BCDEFGHIJKLMNOPQRSTUVWXYZA
shift = 1
matrix = [ source[(i + shift) % 26] for i in range(len(source)) ]

def coder(thistext):
    ciphertext = []
    control = 0

    for x,i in enumerate(input_text.upper()):
        if i not in source: 
            #If the symbol is not in our reference alphabet, we simply print it
            ciphertext.append(i)
            continue
        else:
            #Wrap around the mykey string 
            control = 0 if control % len(mykey) == 0 else control 

            #Calculate the position C[i] = (M[i]+K[i]) mod len(M)
            result = (source.find(i) + matrix.index(mykey[control])) % 26

            #Add the symbol in position "result" to be printed later
            ciphertext.append(matrix[result])
            control += 1

    return ciphertext

def decoder(thistext):
    control = 0
    plaintext = []

    for x,i in enumerate(code_text.upper()):
        if i not in source: 
            #If the symbol is not in our reference alphabet, we simply print it
            plaintext.append(i)
            continue
        else:
            #Wrap around the mykey string 
            control = 0 if control % len(mykey) == 0 else control 

            #Calculate the position M[i] = (C[i]-K[i]) mod len(M)
            result = (matrix.index(i) - matrix.index(mykey[control])) % 26

            #Add the symbol in position "result" to be printed later
            plaintext.append(source[result])
            control += 1

    return plaintext

# Print results
print("Key: {0}".format(mykey))
print("\nDecode text:")
print("-> Input text: {0}".format(input_text))
print("-> Coded text: {0}".format(''.join(coder(input_text))))

# Print results
print("\nDecode text:")
print("-> Input text: {0}".format(code_text))
print("-> Decoded text: {0}".format(''.join(decoder(code_text)).lower()))

Code stored in GitHub

The script is pretty basic and simple to understand. There are two functions, and the key part is the calculation of result using the math formula shown above.

And see the result:

$ python Vigenere_cipher_mod.py
Key: WHITE

Decode text:
-> Input text: en un lugar de la mancha de cuyo nombre no quiero acordarme
-> Coded text: AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM

Decode text:
-> Input text: AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM
-> Decoded text: en un lugar de la mancha de cuyo nombre no quiero acordarme

There are tons of references about how to break this code on the internet. I found these two very interesting:

Later

PS: I hate markdown. Just a little bit...

Simple Vigenere cipher in Python (2)

psgonza — Wed, 08 Mar 2017 13:13:17 +0200

See: - Part 1/3 - Part 3/3

Just a small update to my previous post about the Vigenere cipher

Following the same approach as in the cipher, I modified a few lines in the script to create the "decoder"... I won't paste the code here, because the script is 95% the same, but I have stored it in GITHUB.

Basically, some variables change:

coder: input_text="en un lugar de la mancha de cuyo nombre no quiero acordarme"
decoder: input_text="AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM"
coder: ciphertext = []
decoder: cleartext = []
coder: print("-> Output text: {0}".format(''.join(ciphertext)))
decoder: print("-> Output text: {0}".format(''.join(cleartext)))

And the logic for the look up changes as well:

In the coder, we replace the letter in the input_text by the one in the matrix[n] :

for x,y in encryption_tuple:
    if source.find(x) == -1: 
        ciphertext.append(x)
    else:
        ref_row = matrix[0].index(y)
        ciphertext.append(matrix[ref_row][source.index(x)])

In the decoder, we look for the position (lets call it "y") of the letter in matrix[n] and replace it by the letter in the position "y" in the alphabet:

for x,y in encryption_tuple:
    if source.find(x) == -1: 
        cleartext.append(x)
    else:
        ref_row = matrix[0].index(y)
        cleartext.append(source[matrix[ref_row].index(x)])

The result is the expected:

-> Key: WHITE
-> Input text: AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM
-> Output text: EN UN LUGAR DE LA MANCHA DE CUYO NOMBRE NO QUIERO ACORDARME

In the next (and last post) about Vigenere, I'll write another simple coder/decoder script but based in the mathematical concept

Laters!

Simple Vigenere cipher in Python

psgonza — Tue, 07 Mar 2017 13:13:17 +0200

See: - Part 2/3 - Part 3/3

I am currently reading "The code book" by Simon Singh, and he just described how the Vigenere cipher works... I am not coding any Python lately, so I have decided to implement it (real quick), not using any algorithm but manually, as someone would have done 300 years ago, preparing a Vigenere square, and then looking up the values in the table.

The Python code is pretty simple:

#!/usr/bin/env python
# Simple Vigenere cipher implementation in Python
import string

mykey="WHITE"
input_text="en un lugar de la mancha de cuyo nombre no quiero acordarme"

ciphertext = []
matrix = []
encryption_tuple= []
row = 0
control = 0

# Alphabet used as reference
source = string.ascii_uppercase

# Creating the Vigenere Square. A 26x26 matrix. 
# In the example provided by the book, instead of using the regular alphabet as reference 
# we shift the items, so the column used as reference doesn't start in A, but in B
for row in range(len(source)):
    matrix.append([ x for i,x in enumerate(source) if i > row ])   
    for i,x in enumerate(source):
        if i <= row: matrix[row].append(x)

# Creating the tuple based on the letter and key. ie:
# ('D', 'W'), ('I', 'H'), ('V', 'I'), ('E', 'T'), ('R', 'E'), ('T', 'W'), ...        
# In case special characters are not considered, this is cleaner:
#   import itertools
#   text=[ x for x in input_text.upper() if x in string.ascii_letters]
#   encryption_tuple = [(x,y) for x,y in zip(text, itertools.cycle(mykey))]
for x,y in enumerate(input_text.upper()):
    control = 0 if control % len(mykey) == 0 else control
    if y in string.punctuation or y in string.whitespace:
         encryption_tuple.append((y,y))
    else:
         encryption_tuple.append((y,mykey[control]))
         control += 1

# Each element y in the tuple is the key in the alphabet matrix
for x,y in encryption_tuple:
    if source.find(x) == -1: 
        ciphertext.append(x)
    else:
        ref_row = matrix[0].index(y)
        ciphertext.append(matrix[ref_row][source.index(x)])

# Print guide
print("-> Reference:")        
print("   " + ' '.join([x for x in source]))
# Printing Vigenere square
print("-> Square:")        
for id,i in enumerate(matrix,1):
    print("{:02d} {}".format(id,' '.join(i)))
# Print results
print("-> Key: {0}".format(mykey))
print("-> Input text: {0}".format(input_text))
print("-> Output text: {0}".format(''.join(ciphertext)))

I stored the code in GITHUB.

In my case, as in the book, I have used "WHITE" as keyword, and the string to cipher is hardcoded in the script (input_text)

The main step is to map the input_text("en un lugar...") with the keyword("WHITE"), so we end up with something like this:

('E', 'W')
('N', 'H')
(' ', ' ')
('U', 'I')
('N', 'T')
(' ', ' ')
('L', 'E')
('U', 'W')
('G', 'H')
('A', 'I')
('R', 'T')
(' ', ' ')
('D', 'E')
[...]

The keyword will be repeated over and over until input_text string finishes
For the sake of clarity, I haven't stripped the white spaces or any other punctuation symbol from the input message, but it makes the cipher (even) weaker.

Those tuples are going to be used to look for the letter to replace the actual symbol. For example: In order to cipher "E", we will use the row 22, as is the one starting by "W", therefore, the letter "E" will be replaced by "A". Next letter is "N", and we will use row 7, so it will be replaced by "U", and so on...

Not very likely to replace opengpg, but it works:

$ python vigenere_cipher.py
-> Reference:
   A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
-> Square:
01 B C D E F G H I J K L M N O P Q R S T U V W X Y Z A
02 C D E F G H I J K L M N O P Q R S T U V W X Y Z A B
03 D E F G H I J K L M N O P Q R S T U V W X Y Z A B C
04 E F G H I J K L M N O P Q R S T U V W X Y Z A B C D
05 F G H I J K L M N O P Q R S T U V W X Y Z A B C D E
06 G H I J K L M N O P Q R S T U V W X Y Z A B C D E F
07 H I J K L M N O P Q R S T U V W X Y Z A B C D E F G
08 I J K L M N O P Q R S T U V W X Y Z A B C D E F G H
09 J K L M N O P Q R S T U V W X Y Z A B C D E F G H I
10 K L M N O P Q R S T U V W X Y Z A B C D E F G H I J
11 L M N O P Q R S T U V W X Y Z A B C D E F G H I J K
12 M N O P Q R S T U V W X Y Z A B C D E F G H I J K L
13 N O P Q R S T U V W X Y Z A B C D E F G H I J K L M
14 O P Q R S T U V W X Y Z A B C D E F G H I J K L M N
15 P Q R S T U V W X Y Z A B C D E F G H I J K L M N O
16 Q R S T U V W X Y Z A B C D E F G H I J K L M N O P
17 R S T U V W X Y Z A B C D E F G H I J K L M N O P Q
18 S T U V W X Y Z A B C D E F G H I J K L M N O P Q R
19 T U V W X Y Z A B C D E F G H I J K L M N O P Q R S
20 U V W X Y Z A B C D E F G H I J K L M N O P Q R S T
21 V W X Y Z A B C D E F G H I J K L M N O P Q R S T U
22 W X Y Z A B C D E F G H I J K L M N O P Q R S T U V
23 X Y Z A B C D E F G H I J K L M N O P Q R S T U V W
24 Y Z A B C D E F G H I J K L M N O P Q R S T U V W X
25 Z A B C D E F G H I J K L M N O P Q R S T U V W X Y
26 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

-> Key: WHITE
-> Input text: en un lugar de la mancha de cuyo nombre no quiero acordarme
-> Output text: AU CG PQNIK HA SI FEJJPT HA JCRS JVUUVA UW JYELZH EYVZWENTM

I'll try to find time to code the decryption part and also to try different ways of implementing it.

By the way, I totally recommend the book!

Later

SFTP oneliner (as SCP)

psgonza — Fri, 10 Feb 2017 12:12:17 +0200

Quick note to self:

Secure Copy (scp) (a one line command very easy to include in bash scripts) can be replaced by a similar Secure File Transfer (sftp) command, but the documentation is not very clear...

For instance, this command to transfer a file to a remote host using SCP:

scp src_file user@remote_host:/remote/path/dst_file

Can be replaced by this one, using SFTP:

sftp user@remote_host:/remote/path/ <<< $'put src_file dst_file'

Example:

# sftp root@192.168.1.100:/tmp/ <<< $'put /tmp/test test_sftp'
Connected to 192.168.1.100.
Changing to: /tmp/
sftp> put /tmp/test test_sftp
Uploading /tmp/test to /tmp/test_sftp
/tmp/test                                          100%    0     0.0KB/s   00:00
#

The idea is taken from a few StackOverflow threads, all the credit to them... I decided to put it together here, so I can find it easily.

Later

2017 resolutions

psgonza — Wed, 11 Jan 2017 11:11:17 +0200

{% img center /img/goals2017.jpg '2017resolutions' %}

Another entry in my personal online notebook (considering the number of visits of this site, I don't think it should be called blog or website)

We are only 11 day into the new year, so it is not too late for my 2017 resolutions. I'd rather post them here than keep them in a piece of paper on my desktop because, apparently, making goals "public" helps to succeed (or I can be publicly shamed at the end of the year)

Here they go:

1) The classics:

Work out more often: At least 3 times per week. No matter if it rains or snows.
Eat (more) healthy: It seems I'll be stuck in Spain for a few months (at least until April), so it will be a good opportunity to prepare some green juices (I bought a blender a couple of years ago, and I think I've turned it on 2 times).
Read (more) books: Goodreads helps me to track my reading habits... And they are far from impressive, around 12 books per year. It doesn't sound too bad, but considering I don't have family responsibilities, it is not enough. I think I should blame Netflix for this :)
Study something new: There are countless online courses nowadays available for free. I've already done a couple of them in edX and I'd like to do more. Coursera has some really interesting courses as well.
Improve my English pronunciation: My English is quite good, but still sounds like a spaniard speaking English.

2) Job related:

Look for a different role/position: I've been doing pretty much the same stuff for way too long. Time to change. Period.
Ramp up my python skills: I am kind of stuck with Python. It is enough for my current role (system engineering in a big telco, not development) but I'd say it is not good enough if I want to do something different (more IT related).
Learn some new languages: Go?
Ansible: I've been playing with Ansible every now and then for a couple of years, creating some playbooks for myself, but nothing fancy. I don't have the opportunity to use it in my current gig, so I'd need to look for a way to include it in my daily work.

3) Hobby related:

Fix my home network: 1 ISP router + 1 AP router + 1 openWRT (as firewall) + 1 NAS + several PC/laptops/smartphones/raspberry pi's = a hell of a mess.
Soldering: I have a bunch of ESP8266 chips laying around in a box. I should put them to good use.
Photo library housekeeping: I have traveled a lot these past years, and I have tons of photos in my NAS that I have not even seen. It needs a cleanup.
Post more (and better): I registered this domain back in 2005... And, even though no one reads this (the number of daily page reads is close to zero), I like to post stuff here. It's a good way to keep things available (thanks to github of course). I'd like to post more stuff, and if possible, more interesting.

I'd say this should be enough for 2017.

\\psgonza

My 2016 in photos

psgonza — Mon, 02 Jan 2017 02:02:17 +0200

(Pretty big pictures, sorry about that)

Saying hello to another year with some pictures... 2016 is been a good year, both personally and professionally:

I've been working in Japan for several months again... Best place for living as expat (IMO)

{% img center /img/2016_japan.jpg 'Japan' %}

April: I never really liked south east Asia that much (too hot for my taste), but I have to say I had a really good time in Vietnam and Cambodia... Looking forward to going back there soon!

{% img center /img/2016_vietnam_cambodia.jpg 'vietnam_cambodia' %}

July: Raising sun at the top of Mt Fuji

{% img center /img/2016_fujisan.jpg 'fuji' %}

August: Hawaii is the place to be. Period.

{% img center /img/2016_hawaii.jpg 'hawaii' %}

December: Vienna (European capital of Xmas) & Salzburg

{% img center /img/2016_austria.jpg 'austria' %}

I hope 2017 is as good as 2016, although I'd like to make some changes...

Happy New Year everyone

\\psgonza

Related: 2015 in photos, 2014 in photos

My books in 2016

psgonza — Sun, 01 Jan 2017 01:01:17 +0200

We just said goodbye to 2016, and as in 2015, here goes my Goodreads summary:

{% img center /img/books2016.gif 'goodreads' %}

Unfortunately, I failed my reading challenge... I read 10 of the 13 books in the first half of the year, but I have been stuck with a Python book for months. And podcasts have not helped much :)

Happy reading!

\\psgonza

Geolocating your holidays

psgonza — Tue, 20 Dec 2016 00:00:00 +0100

Hi there

Tinkering with Python once again... Just for the sake of it :)

The idea is to generate a web page containing a Google Map with marks of your photo timestamps, geolocated in the map. I'm pretty sure Google Photos is able to do something similar (and better), but where's the fun in that?

For instance, I spent a few days in Kauai this summer, and this is the resulting map:

{% img center /img/20-12-2016-geotagging_map.gif 'Kauai photos timestamp' %}

Each mark is a photo I took, and you can see its timestamp moving the mouse over the marker...

Nothing fancy, but in case it is useful to you, this is the code:

#!/usr/bin/env python
"""
Giving a directory as input:
1) the script will extract the dates and GPS coordinates of the NEF files
2) will add a marker in a Map 
3) print the HTML file

Dependencies: 
- EXIF data has to contain GPS tags
- exifread module installed
- Google Map API credentials
"""
import exifread,sys,glob

G_API_KEY = '<GOOGLE_MAP_API_TOKEN>'

html="""<!DOCTYPE html>
<html>
<head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no">
    <meta charset="utf-8">
    <title>Complex icons</title>
    <style>
      html, body {
        height: 100%;
        margin: 0;
        padding: 0;
      }
      #map {
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="map"></div>
    <script>

function initMap() {
  var map = new google.maps.Map(document.getElementById('map'), {
    zoom: 10,
    center: {lat: xxxLATxxx, lng: xxxLONGxxx}
  });

  setMarkers(map);
}  

xxxCAPTURESxxx

function setMarkers(map) {
  for (var i = 0; i < captures.length; i++) {
    var capture = captures[i];
    var image = 'data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20width%3D%2238%22%20height%3D%2238%22%20viewBox%3D%220%200%2038%2038%22%3E%3Cpath%20fill%3D%22%23808080%22%20stroke%3D%22%23ccc%22%20stroke-width%3D%22.5%22%20d%3D%22M34.305%2016.234c0%208.83-15.148%2019.158-15.148%2019.158S3.507%2025.065%203.507%2016.1c0-8.505%206.894-14.304%2015.4-14.304%208.504%200%2015.398%205.933%2015.398%2014.438z%22%2F%3E%3Ctext%20transform%3D%22translate%2819%2018.5%29%22%20fill%3D%22%23fff%22%20style%3D%22font-family%3A%20Arial%2C%20sans-serif%3Bfont-weight%3Abold%3Btext-align%3Acenter%3B%22%20font-size%3D%2212%22%20text-anchor%3D%22middle%22%3E' + capture[3] + '%3C%2Ftext%3E%3C%2Fsvg%3E';
    var marker = new google.maps.Marker({
      position: { lat: capture[1], lng: capture[2] },
      map: map,
      icon: image,
      title: capture[0],
      zIndex: capture[3]
    });
  }
}
    </script>
    <script async defer
        src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY&signed_in=true&callback=initMap"></script>
  </body>
</html> """ 

def formatme(mydata,reference):
    coordenates=(str(mydata).replace("[","").replace("]","").replace(" ","").split(","))

    d=coordenates[0]
    m=coordenates[1]
    s=coordenates[2]

    ref = str(reference).strip()

    d = int(d.split("/")[0]) / int(d.split("/")[1]) if "/" in d else int(d)
    m = int(m.split("/")[0]) / int(m.split("/")[1]) if "/" in m else int(m)
    s = int(s.split("/")[0]) / int(s.split("/")[1]) if "/" in s else int(s)

    coor = d + (m / 60.0) + (s / 3600.0)
    return coor if ref == "N" or ref == "E" else -coor

def find_pics(pics_path):
    #Some more logic could be added here
    files = glob.glob(pics_path + "/*.NEF") 
    files.sort(key=lambda x: os.path.getmtime(x))
    return files

def get_GPS_data(id,path):
    try:
        #Get the EXIF tags
        with open(path, 'rb') as f:
            tags = exifread.process_file(f,details=False)
    except Exception as e:
        data = ["skip"]

    #Check we got the GPS tag data
    if "GPS GPSLongitude" not in tags and "GPS GPSLongitudeRef" not in tags:
        data = ["skip"]
    else:
        data = [path,"{0}".format(tags["EXIF DateTimeOriginal"]),formatme(tags["GPS GPSLatitude"], \
                tags["GPS GPSLatitudeRef"]),formatme(tags["GPS GPSLongitude"], tags["GPS GPSLongitudeRef"]),id]

    return data

if __name__ == "__main__":
    try:
        path=sys.argv[1]
    except:
        print("Usage: " + sys.argv[0] + " <path>")
        sys.exit(1)

    #List of pictures to process
    pics_list=find_pics(path)

    #List of pics, dates, lat, long, id
    captures = []
    for _, pic in enumerate(pics_list):
        item = get_GPS_data(_, pic) 
        if "skip" not in item: captures.append(item)

    # Create the JS code with the list of coordenates
    html_loop = "var captures = ["
    for capture in captures:
        html_loop = html_loop + "".join("['" + str(capture[1]) + "'," + str(capture[2]) + "," + str(capture[3]) + \
                    "," + str(capture[4]) + "],\n")
    html_loop = html_loop[:-2] + "]; "

    #Update center LAT/LONG  coordenates
    html = html.replace("xxxLATxxx",str(captures[0][2]))
    html = html.replace("xxxLONGxxx",str(captures[0][3]))
    #Update list of points
    html = html.replace("xxxCAPTURESxxx",html_loop)
    #Update API KEY
    html = html.replace("YOUR_API_KEY",G_API_KEY)

    print(html)

Stored in Github

The script is so simple that I am not even opening a file to write the output... Feel free to do it yourself, or redirect the output to a file:

# python extract_geolocation_batch_to_html.py > mymap.htm

Take care out there...

\\psgonza

Spam via Google Analytics (update)

psgonza — Fri, 16 Dec 2016 00:00:00 +0100

I am not saying Russia has anything to do with the US presidential elections results, but checking Google Analytics, the referral spam is kind of "funny"...

{% img center /img/16122016_spam.JPG 'spam' %}

And the location:

{% img center /img/16122016_spam2.JPG 'spam' %}

Apparently a lot people followed their advice and voted for Trump.

Later

Trying to minimize tracking in mobile platforms

psgonza — Wed, 14 Dec 2016 00:00:00 +0100

Random thoughts about how to reduce your "big data footprint"

1) Social networks

I have managed to reduce my "wasted" time in social networks to the bare minimum...

Earlier this year I "quit" G+ (it's a pitty, it had a lot of potential, but it ended up badly)
I uninstalled FB app from my smartphone months ago, so I use the browser to log in FB (I keep Twitter app installed though)
I spend just a few minutes in FB/Twitter every day. Pretty low profile for today's standards, I guess.

But the tracking ads in FB are driving me crazy.

Quick examples:

a) I was having a discussion with some friends about portable espresso machines (yes, they do exist), so I googled it and went on with my life. Next time I logged in FB, there it was... Portable espresso machine:

{% img center /img/photo_2016-12-14_11-38-13.jpg 'ads' %}

b) Another discussion about smart trackers, another search and 15 minutes later I am seeing promoted posts in Facebook.

Probably I was using Chrome Browser and Google Search for this, so not only FB is to blame... And don't get me wrong, I don't mind Mark making loads of money sharing (should I say "selling"?) people's info (life?), but just not mine...

As I said, I already removed Facebook Android application from my phone (and you should probably do the same) and I was using my main browser (Chrome) to log in, but third-party cookies are not helping here... I have decided to use Firefox browser mainly for Facebook. It was already installed in my smartphone, so I simply created a shortcut to the FB url in the "desktop"

If you are not a heavy FB user, this will do the job... It is better than the FB lite application, and you will be able to isolate your browsing history from your FB session (I'm sorry Mark)

2) Google ads + location

It's not easy to stay away from Google when you are using Android devices... I am a happy CyanogenMod user (both in my OnePlus One and my old Samsung S3) but Google services are the core of the OS, and, truth being told, they tend to make your life easier. It comes with a price, of course: "If You're Not Paying for It, You're the Product"

There are a lot of information in the Internet about Google services alternatives such as leavegooglebehind.com but it is not an easy task (see Bryan's post series in Networkworld). I haven't gone that far yet but there are some easy alternatives to Google apps that do the job for me:

Using different internet browsers such as Firefox (not "instead of" but together with Chrome)
Duckduckgo as primary search engine (instead of Google Search)
Telegram as IM client (instead of Hangouts/Allo)
Piktures as gallery app (instead of Photos)
Dropbox as shared storage (instead of Drive)

Some other applications such as GMail, GMaps, Youtube, GDocs, etc are not so easy to replace, and I wouldn't look for an alternative unless you are really decided to live out of Google's kingdom.

And one step further: disable GPS, disable WLAN when not connected, remove old SSIDs from your WLAN, disable Google history, and location services... There are many things to do if you are concerned about your privacy. However, being connected 24/7 means there will be a track of your activities/locations somewhere, like it or not.

Trying to keep the same level of privacy as in 1990's, when people were unable to reach you if you weren't at home, is kind of impossible nowadays. Embrace it or go find a nice cave because you are going to have a hard time in the next 5-10 years.

Let's be careful out there

//psgonza

PS: And rest assured that not only companies like Google, FB, Apple, Amazon, etc are willing to track you... Bad guys and goverments have a lot of interest in your digital life as well...

Spam via Google Analytics

psgonza — Fri, 07 Oct 2016 00:00:00 +0200

I have to give it to them... People are pretty damn creative... Email spam, Whatsapp spam, and why not... Referral spam!

{% img center /img/spam_via_analytics.JPG 'spam' %}

People should put all this effort and time in making the world a better place

Minimal Cygwin Python data science installation

psgonza — Sat, 01 Oct 2016 00:00:00 +0200

A few days ago a made a fresh installation of Cygwin64 in order to tinker with Pandas (just for fun) but instead of installing Anaconda, I decided to install each component manually... I will write it here because I'm sure I will need it in the future (and it might be handy for someone else).

First things first, package management in Cygwin is a bit of a pain without apt-cyg... So download it from here, save it as "apt-cyg" in your $HOME and install it:

install apt-cyg /bin/

Running telegram-cli in cygwin

psgonza — Fri, 05 Aug 2016 00:00:00 +0200

Telegram is a really interesting messaging application... It is secure (well, let's say secure enough), fast and multiplatform... Even there is a CLI client!

I tried to compile the telegram-cli in my cygwin64 earlier today following their documentation, and I came across a couple of issues.

1) Patching failed:

I couldn't patch Makefile and Hunk #1 failed in loop.c as well:

20:34:26@tg$ patch -p1 < telegram-cli-cygwin.patch
patching file Makefile
Hunk #1 FAILED at 4.
1 out of 1 hunk FAILED -- saving rejects to file Makefile.rej
patching file loop.c
Hunk #1 FAILED at 383.
Hunk #2 succeeded at 737 with fuzz 2 (offset 103 lines).
1 out of 2 hunks FAILED -- saving rejects to file loop.c.rej

Probably the patch file is not updated with the latest telegram-cli changes, so I dediced to patch it manually...

I updated the first lines of Makefile with the changes stated in the patch file:

1 srcdir=.
2
3 CFLAGS=-g -O2
4 LDFLAGS= -L/usr/local/lib -L/usr/lib -L/usr/lib -L/usr/lib
5 CPPFLAGS= -I/usr/local/include -I/usr/include -I/usr/include  -I/usr/include/python3.4m -I/usr/include
6 DEFS=-DHAVE_CONFIG_H
7
8 COMPILE_FLAGS=${CFLAGS} ${CPFLAGS} ${CPPFLAGS} ${DEFS} -Wall -Werror -Wextra -Wno-missing-field-initializers 
   -Wno-deprecated-declarations -fno-strict-aliasing -fn      o-omit-frame-pointer -ggdb -Wno-unused-parameter
9 EXTRA_LIBS=-ljansson -lconfig -lz -levent -lm   -lreadline -llua-5.2  -lpython3.4m -lssl -lcrypto
10 LOCAL_LDFLAGS=-ggdb -levent ${EXTRA_LIBS} -ldl -lpthread -lutil
11 LINK_FLAGS=${LDFLAGS} ${LOCAL_LDFLAGS}

Since loop.c was partially patched already, I tried to manually fix Hunk #1, but nothing worked, so I guess it is no longer neccessary...

I managed to generate the binary file after patching Makefile... But it still refused to work.

2) Error assertion "0" failed: file "tgl/mtproto-utils.c", line 101, function: BN2ull

There were no compilation errors, but there was something wrong in tgl/mtproto-utils.c:

21:00:14@tg$ bin/telegram-cli -k tg-server.pub
Telegram-cli version 1.4.1, Copyright (C) 2013-2015 Vitaly Valtman
Telegram-cli comes with ABSOLUTELY NO WARRANTY; for details type `show_license'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show_license' for details.
Telegram-cli uses libtgl version 2.1.0
Telegram-cli includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit. (http://www.openssl.org/)
I: config dir=[/home/user/.telegram-cli]
[/home/user/.telegram-cli] created
[/home/user/.telegram-cli/downloads] created
> SIGNAL received
No libexec. Backtrace disabled
assertion "0" failed: file "tgl/mtproto-utils.c", line 101, function: BN2ull

The solution seems to be remove (or comment out) lines 101 and 115 in tgl/mtproto-utils.c:

[...]
101     //assert (0); // As long as nobody ever uses this code, assume it is broken.
[...]
115     //assert (0); // As long as nobody ever uses this code, assume it is broken.
[...]

After that, run make again, and execute it:

21:13:09@tg$ bin/telegram-cli -k tg-server.pub
Telegram-cli version 1.4.1, Copyright (C) 2013-2015 Vitaly Valtman
Telegram-cli comes with ABSOLUTELY NO WARRANTY; for details type `show_license'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show_license' for details.
Telegram-cli uses libtgl version 2.1.0
Telegram-cli includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit. (http://www.openssl.org/)
I: config dir=[/home/user/.telegram-cli]
>

I hope it helps!

\\psgonza

Tamron SP AF10-24mm

psgonza — Mon, 28 Mar 2016 11:11:11 +0200

I am not a pro photographer and don't like to spend a fortune in gear for my camera, but I've always wanted to buy an ultrawide lense...

I had the opportunity to buy the Tamron SP AF10-24mm for a decent price, and so far I am liking it... See a couple of pics I took a few days ago:

{% img center /img/DSC_2861.jpg 'Shibuya crossing'%}

{% img center /img/DSC_2618.jpg 'Zenko-ji Nagano'%}

{% img center /img/2016-03-21.jpg 'Matsumoto Castle'%}

Maybe not the best uw lense out there, but good enough for the price

\\psgonza

We love you Bash

psgonza — Tue, 09 Feb 2016 11:11:11 +0100

We love you Bash… but sometimes you are so damn slow

I'm sure this has happened to you before… I had to create an ldif file containing 70.000 entries like these ones:

dn: user=XXXXXXXXXX,rootdn=com
changetype: modify
replace: attr1
attr1: qwerty1
-
replace: attr2
attr2: {"reporting": [{"name":"Total","reportingLevel":"total","subscription":"A", "time":500,"reset":{"main":"30 ","time":"30 "}}}]}

(the real data was a little bit larger, this is just an example)

It was supposed to be something really quick and simple, one of those one-liners you write in 3 seconds… I said to myself: bash + loop + echo + redirection to file = WIN

Well, it proved me wrong :

The quick and dirty code to be executed in the shell:

for i in `seq 1 70000`;
do
echo -e "dn: user=XXXXXXXXXX,rootdn=com
changetype: modify
replace: attr1
attr1: qwerty1
-
replace: attr2
attr2: {\"reporting\": [{\"name\":\"Total\",\"reportingLevel\":\"total\",\"subscription\":\"A\", \"time\":500,\"reset\":{\"main\":\"30 \",\"time\":\"30 \"}}}]}
"  >> modify.ldif
done

So I issued that command, and went out for a bite… 30 minutes later, the fraking thing was still running!!! How was that even possible…

I realized I was opening the file every time I wrote a line, a nonsense… so I just stopped the script and tweaked it a little bit… Result: pretty much the same… Here's some figures with only 5000 iterations per loop with different redirections:

Initial redirection:

for i in `seq 1 5000`; do
…
> "  >> modify.ldif; done
real    4m17.606s
user    1m4.028s
sys     1m35.850s

Only one redirection outside of the loop:

for i in `seq 1 5000`; do
…
> " ; done >> modify.ldif
real    4m30.218s
user    1m8.496s
sys     1m41.390s

I also tried to use a file descriptor, just for the sake of it, but it has no impact:

exec 4>> modify.ldif
for i in `seq 1 5000`; do
…
> "  >&4; done
real    4m32.197s
user    1m10.248s
sys     1m39.294s

And using the FD outside of the loop:

for i in `seq 1 5000`; do
…
> "; done >&4
real    4m31.478s
user    1m11.740s
sys     1m38.678s

So all in all, as you can see, Bash IO redirection performance, basically sucks.

A 20 lines python script (with no optimization whatsoever) is able to generate the whole ldif file (creating 70.000 entries to modify) in less than 1 second:

#!/usr/bin/env python

data="""dn: user=XXXXXXXXXX,rootdn=com
changetype: modify
replace: attr1
attr1: qwerty1
-
replace: attr2
attr2: {\"reporting\": [{\"name\":\"Total\",\"reportingLevel\":\"total\",\"subscription\":\"A\", \"time\":500,\"reset\":{\"main\":\"30 \",\"time\":\"30 \"}}}]}
"""

with open("modify.ldif",'w') as outputF:
    for iter in xrange(70000):
        outputF.write(data.replace("XXXXXXXXXX ",str(iter)))

Executing it:

# time python gen_ldif.py
real    0m0.506s
user    0m0.108s
sys     0m0.052s

How about that? Sometimes it's just better to use the right tool for the right job...

So yeah, we love you Bash… Even though you are really slow sometimes.

\\psgonza

Moving to SSL/TLS

psgonza — Sat, 30 Jan 2016 02:03:04 +0200

10 minutes... That's all it takes to configure HTTPS thanks to Let's Encrypt. So from now on, this blog will be using https by default..

There is a lot of information about how to configure it available in Internet (you can find a usefull piece of information here) so I won't even try to add anything new.

Let's Encrypt certs have a ninety-day lifetime, so make sure you renew your certifications each 3 months (some examples here).

As a final note, if you are running a static blog generator as myself, don't forget to modify your config file and replace "http://" by "https://".

\\psgonza

Docker and Kubernetes Workshop

psgonza — Sun, 17 Jan 2016 01:17:17 +0200

Yesterday I attended a very interesting workshop in ShuttleCloud offices in Madrid about Docker and Kubernetes...

I'd say most of the people already knew a little bit about Docker (at least basic knowledge), but it was the first time with Kubernetes for most of us... It was a very good introduction to the technology, and we spent a few hours playing around with pods,services and replicationcontrollers in our own cluster (it is pretty simple to setup a small environment using Vagrant or Google Cloud Engine.

I'd love to attend to more whole-day events like this one (lunch was a plus, but it is not needed whatsoever), since it is a great way to learn/improve our technical skills and helps to build a "community" (I saw some familiar faces from Python meetups in the workshop)

This is a picture of all the participants:

{% img center /img/kubernetes_workshop.jpg 'workshop' %} (taken from Shuttlecloud twitter account)

Thanks to Ivan and Alex for sharing their knowledge, and of course, to jobandtalent for providing the lunch!

\\psgonza

Free ranting about iPhone6

psgonza — Thu, 14 Jan 2016 01:02:03 +0200

Here's a few things about Apple iPhone6 that have been bugging me lately:

1 - I was asked to delete all the photos from an iPhone6... I was hoping it was going to be a matter of connecting the phone to my laptop (MBP), click here, click there, select all, delete and you're done... Well, apparently, it is easier to put a rocket in Mars than deleting all the freaking photos from an iPhone6. Shame on you Apple... you made my life miserable for 1 hour (I had to delete all the photos from the phone itself.... 5467 photos!)

2 - Playing around with the same phone, I pressed the home button (btw, who can live with only one button?? for everything!! are you crazy?) for long enough to invoke Siri, with the phone locked... And she answered to me!!! Let me repeat it.. I long-pressed the home button, I asked Siri to "call mum" (not my mum obviously because it wasn't my phone) from another person locked phone... I would say it is a huge security flaw to allow anyone to make calls that way. Shame on you Apple, again.

\\psgonza

PS1: I am not an Android/Google fanboy... I own a MBP and I cannot complain, it just works.

PS2: Probably these two things I mentioned are not new if you are an iPhone user, which I am not...

PS3: Ranting is free :P

My books in 2015

psgonza — Fri, 01 Jan 2016 01:01:16 +0200

Goodreads is a great social media platform for reading lovers... It allows you to discover new books, keep track of your reading (it is even integrated with Amazon Kidle) or recommend books to your friends... If reading is one of your hibbies, you should check it out.

It's far from impressive I know (some of my friends are able to read more than 40 books in a year.. don't ask me how) but, despite Netflix, I managed to read 14 books (and I have another two books "in progess"):

{% img center /img/goodreads2015.jpg 'goodreads' %}

My recommendations:

1) The martian

2) I am pilgrim

3) Ready player one

Happy reading...

\psgonza

Bye 2015 && Hello 2016

psgonza — Thu, 31 Dec 2015 15:16:17 +0200

Again, it is time to say goodbye to another year... This time to 2015:

Japan:

{% img center /img/2015japan.jpg 'japan' %}

China:

{% img center /img/2015china.jpg 'china' %}

Maldives.. By air, land and sea:

{% img center /img/2015maldives.jpg 'maldives' %}

London:

{% img center /img/2015london.jpg 'london' %}

(50% business/50% leisure)

Happy New Year 2016!!!

\psgonza

Mail notification system in 2 minutes

psgonza — Sun, 27 Dec 2015 11:11:11 +0100

Quick post... In case you'd like to set up a quick mail notification system for your linux machines, keep reading, otherwise, have a merry Xmas...

I'd like to receive a notification everytime there is a sucessfull ssh login in my VPS, and swatch + ssmtp make it possible very easily... The idea is to receive a notification in my email every time there is a line in /var/log/auth.log containing the strings "sshd" and "Accepted password". ie:

Dec 24 12:42:06 bynario sshd[3258]: Accepted password for myuser from 95.11.176.11 port 31738 ssh2

NOTE: Using your gmail account might give you some troubles... I decided to use a different one.

Install the sw:

sudo apt-get install ssmtp swatch

Make sure no one else can read your email password:

sudo chown root:mail /etc/ssmtp/ssmtp.conf
sudo chmod 640 /etc/ssmtp/ssmtp.conf
sudo usermod -a -G mail myuser

(myuser is the name of my local user)

Create /etc/swatch.conf and add the pattern and action:

watchfor /sshd.*Accepted password/
    exec echo "Subject:Bynario.com SSHD accepted password\n\n$_\n" | /usr/sbin/ssmtp <your_email>

what does it do? It will be tailing the auth.log file looking for any string matching "sshd.*Accepted password" and in case it finds something, it will send an email to with the subject "Bynario.com SSHD accepted password" and the line matching the filter in the email body.

Let's define the email account... Edit /etc/ssmtp/ssmtp.conf:

root=<your_email>

UseTLS=YES
UseSTARTTLS=YES
AuthMethod=LOGIN

mailhub=<your_snmp_server.com>:587
hostname=<your_server.com>

AuthUser=<your_email>
AuthPass=<your_password>

FromLineOverride=YES

(The TSL options will depend on your email provider)

Edit /etc/ssmtp/revaliases:

root:<your_email>:<your_snmp_server.com>:587
myuser:<your_email>:<your_snmp_server.com>:587

And finally, start swatch:

sudo swatch --config-file=/etc/swatch.conf --tail-file=/var/log/auth.log --tail-args=--follow=name --daemon

It would be a good idea to create an rc script or add that line to /etc/rc.local, so it is automatically started if the system is rebooted:

bynario:~$ cat /etc/rc.local
/usr/bin/swatch --config-file=/etc/swatch.conf --tail-file=/var/log/auth.log --tail-args=--follow=name --daemon
exit 0

And that's it... You should have an email in your account every time there is a new ssh connection... pretty handy.

\psgonza

2015 The year of the podcasts

psgonza — Fri, 25 Dec 2015 11:11:11 +0100

Well, I know I might be terribly late on this, but for me, 2015 has been the year I have (re)discovered the podcasts...

According to Wikipedia podcasts are around 10 years old, but I never really put too much attention into them until this year. I have been an avid user of RSS feeds since long time ago, so I never really had the need of listening to other people's opinions when you can read a blog post about it (way faster than listening).

But, as it happens to almost everybody, I reached a point in life where I don't have time to spare in reading +100 post every day... Adulthood sucks, I guess (and I don't have family responsabilities, I cannot imagine how people survive to parenting.)

Earlier this year I was in a business trip in Tokyo when spring came, and with the nice weather I started to take long walks (one of the best ways to disconnect from work and with a lof of health benefits according to some studies).

First I tried to take my Amazon kindle with me, but it slows you down and you end up walking really slowly. Then I tried with audiobooks, not a bad solution at all, but not cheap (more expensive than kindle books for sure), and I'd rather read at the same time I enjoy a cup of coffee...

So not sure why, podcasts came back to my mind... I had already listen some of them in my laptop every now and then, but never really bothered to use my Android phone as a media player. And they were exactly what I was after:

1 - They are not a replacement for the blog post, but it might be the closest thing.

2 - You can listen to them while you walk/run/conmute/whatever.

3 - They are really cheap (some of them have sponsors or ask you for donation).

4 - There are several nice Android apps that handle the subscrition for you.

The number of available podcast is incredible. In my case I am mostly interested in technology, but I am sure that, on any given topic you are insterested in, there are several podcasts out there waiting for you (speaking english makes it easier of course).

This is my current Podcast Addict subscription list:

Arrested DevOps
The Ship show
DevOps Cafe
The Food Fight Show
Bad Voltage
This Week In Tech (TWiT)
This Week in Google
Danielmiessler
Floss Weekly
Pauldotcom security weekly
Talk Python To Me
Python Test Podcast
Podcast.init
From Python import podcast (Not active)
Serial podcast (Season 2 just started. It is more like a radio serial from the 1960's)
In our time: Science

Give them a try, because some of them are really really good.

\pgonza

Spotify playlist tracks

psgonza — Sat, 28 Nov 2015 11:28:15 +0100

I've never really understood why Spotify doesn't allow me to export my playlists into text file... I guess there is a reason (I bet they have a good one), but I just don't see it.

Anyway, most of the times it's quicker to try to fix the problem yourself than waiting for other to fix it...

Since Spotify has a nice API (with no auth needed) for getting tracks information available, I created a small script that would iterate over the songs in my playlist, and printout the information (Artist - Album - Song), and I though it might be usefull for someone else.

First of all, get the list of Spotify URIs from all the songs in your play list:

Navigate to your playlist in Spotify, and select all the songs (CRL-A).
Click "Copy Spotify URI" from the context menu.
Paste in a new file ie: "myplaylist.txt"

The file should look like this one:

$ cat myplaylist.txt
spotify:track:1OGFtaUgHAQjtSk7mhDwr9
spotify:track:39J10NL0mFTAdJbapoo2rC
spotify:track:3G0EKJZy0j3rMG077UawaC
spotify:track:5VdVaUBgj7cBTKplgaIhKu
spotify:track:2EBuFjexd3S3wc9m4Rerh8
...

The idea is to call Spotify API for each track id (spotify:track:<track id>) in the myplaylist.txt file, get the json response and parse the artist,album and song information... As simple as that.

See below the script:

#!/usr/bin/env python
# Getting track details from a list of Spotify URLs via spotify API
# Input file containing the list of URLs mandatory
# Date: 26 Nov 2015

import json, urllib2, sys
from time import sleep

#Checking input file
if len(sys.argv) != 2:
    print("Error: Input file mandatory")
    sys.exit(0)
else:
    input_file = sys.argv[1]

#Spotify track API
api_url = "https://api.spotify.com/v1/tracks/"

#track dictionary
mytracks = []

#Open input file
try:
    fd = open(input_file,"r")
except Exception as e:
    print("Error opening %s" % input_file)
    print(e)
    sys.exit(0)

#getting track details from uri
def get_track_data(url):

    request = urllib2.Request(api_url + url)
    request.add_header('User-Agent', 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)')
    request.add_header('Content-Type','application/json')

    try:
        response = urllib2.urlopen(request)
    except Exception as e:
        print("\nError connecting to spotify API looking details for uri %s" % url)
        return False

    try:
        data = json.load(response)
    except Exception as e:
        print("\nError handling json object looking details for uri %s" % url)
        return False

    try:
        mytracks.append({'album':str(data['album']['name']),'artist':str(data['artists'][0]['name']),'track':str(data['name'])})
        sys.stdout.write(".")
        sys.stdout.flush()
    except Exception as e:
        print("Error getting details for uri %s" % url)
        return False

########
# MAIN #
########

sys.stdout.write("Iterating through tracks in " + input_file)
sys.stdout.flush()

#Calling to get_track_data for each uri in the input file
for track in fd:
    get_track_data(track.split(":")[2])
    #I don't need to DDoS spotify servers :)
    sleep(2)
print("done!")

#Printing the header when the dict has been populated
print("\nArtist - Album - Track")
print("----------------------")

#Printing the results. By default, ordered by 'artist', but it can be ordered by 'album' or 'track'
for track in sorted(mytracks,key=lambda k: k['artist']):
    print("%s - %s - %s" % (track['artist'],track['album'],track['track']))

#close input file
fd.close()

A couple of things:

1) By default, the script is ordering the result by "artist", but it can be easily changed to "album" or "track" in this line:

for track in sorted(mytracks,key=lambda k: k['artist']):

--> for track in sorted(mytracks,key=lambda k: k['album']):

--> for track in sorted(mytracks,key=lambda k: k['track']):

2) I have decided to take it easy with Spotify servers, so I added sleep 2 between each query.

3) No concurrency, multi-threading, etc... I don't plan to run the script on regular basis, so no need to spend more time on this.

Execution example:

$ python spotify_playlist.py myplaylist.txt
Iterating through tracks in myplaylist.txt......done!

Artist - Album - Track
----------------------
Adema - Adema - Giving In
Alice Cooper - The Best Of Alice Cooper - Poison
Alice In Chains - Facelift - Man in the Box
Alice In Chains - Jar Of Flies - Rotten Apple
Alice In Chains - Jar Of Flies - Nutshell
Alien Ant Farm - Anthology - Smooth Criminal

I know, I know... Only great hits in my playlist :)

Does knowing all the idioms/built-in functions make you a better coder?

psgonza — Wed, 11 Nov 2015 11:11:11 +0100

(Disclaimer: I do not code for a living...)

I like coding in Python, it is my go-to language for almost everything nowadays. Easy to learn, pretty popular and fast enough, so no complains, but sometimes python idioms and built-in function/methods drive me nuts…

I understand you cannot be a good python developer if you don’t know your language, but unless you write python code 24/7, I think we shouldn’t focus on writing 100% pythonic code…. It is not worth it.

Let me put a very basic example. Not sure if you know about checkio, a game/community/platform for learning (or improving) python (100% recommended by the way), where you have hundreds of “missions” (tasks to be solved by using Python), and you can share your solutions with the rest of the community… This is the most interesting part by far, because you can compare your code, and (in my case) realize how bad it is :)

For example, one of the missions asks you to write the code for transposing any given matrix, which could be easily achieved in 7 lines of code if you understand the concept:

Python:

matrix =[[1,2,3], [4,5,6], [7,8,9]]
transpose = []
for j in range(len(matrix[0])):
    row=[]
    for i in matrix:
        row.append(i[j])
    transpose.append(row)

Yes, the code is simple, not optimal, and can be improved a lot, but it can be translated to C in 1 minute:

int j = 3;
int k = 3;
int c, d;
int matrix[3][3]={{1,2,3},{4,5,6},{7,8,9}};
int transpose[3][3];

for (c = 0; c < j; c++)
  for( d = 0 ; d < k ; d++ )
     transpose[d][c] = matrix[c][d];

On the other hand, if I were a python developer, I could come up with something like this:

Python using built-in functions:

matrix =[[1,2,3], [4,5,6], [7,8,9]]
transpose = list(map(list, zip(*matrix)))

(solution by PositronicLlama)

Ok, I love one-liners as much as the next guy and I agree that is a great Pythonic answer… But it makes me wonder:

Is that code easy to read and understand? I don’t think anyone with basic/average python skills can understand it.
When performance is not an issue, why not writing a small piece of code instead of relying in a function you don't even know how works? I am not a python developer (truth being told, I am not even a developer), so for me it is more interesting to understand how to solve a problem than solving it… Because I can use the same approach in any other language if needed.
Are we better developers if we use list/map/zip in the same line? Probably the short answer is yes. I think there is no such thing as “generalist developer”, most of people know a lot about just one language, and if you are really (really) good, you could know 2 or maybe 3 , but that is not that common… So yes, if you want to be a “pr0” python programmer, you would need to know all these python idioms, built-in functions and modules that makes Python so great.

For the “n00b”/average programmers as myself, yeah, they are cool, save a few lines and are damn fast so I use some of them, but you know, it is not that important.

Quick Thought III - Mobiles at home

psgonza — Fri, 30 Oct 2015 10:10:10 +0100

{% img center /img/DSC_1869-001.jpg 'mobiles' %}

Those are the mobiles at my place at the moment (only half of them are mine), all of them are in "working" condition, but 90% of them are basically useless... I keep all the old Nokia phones just as a piece of history.

A few things come to mind:

No way we can sustain this level of electronic wasting. And recycling just means dumping this e-waste someplace in Africa
The smarter the cellphone, the dumber the people.
Blackberry had the best keyboard ever, until big screens came out and swiftkey took the throne.
I don't need a faster/bigger phone. I need a longer battery life!
What the hell happened to you Nokia??

As God is my witness (thanks for the quote Scarlett), I'll try not to buy any other smartphone as long as my OnePlus One does the job

UPDATE Aug 25th 2017 I kept my word!!! Sort of :)

I haven't bought a new phone, but I am using someone else's iphone 6... Reusing is good!

Quick Thought II - Babel 2.0

psgonza — Sat, 10 Oct 2015 10:10:10 +0200

Wouldn't it be easier if we could agree on using just one (ok, let's say two, Apple still think they are different than the rest) instant messenger application???

{% img center /img/Screenshot_2015-10-09-08-44-58.png 'babel' %}

Embedding modules in Python scripts

psgonza — Mon, 21 Sep 2015 10:10:10 +0200

I am a huge fan of pexpect. For those of you who don't know it, it is a module "based" on the same idea as Expect but in Python. As its author said:

I loved Expect, but I hated TCL, so I wrote this 100% pure Python module that does the same thing

Noah Spurrier

I think it's a great tool for ssh automatization whenever priv/pub keys are not an option.

In my case, since I usually work in isolated systems where I cannot even install local python mudules, and given that I am using just a small part of all the funcionalities provided by this module, I came up with the idea of embedding an old version of pexpect module into my scripts, so I don't have to worry about SCP'ing several files each time I need to run any of them.

The idea is to embed the base64 code of the pexpect.py module into the script, and then create the module on the fly whenever the script is executed (if it is not there already).

Simple stuff.... This is the procedure for Python 2.x:

1) Get the base64 code of the module:

LOCAL $ gzip -c pexpect.py | base64
H4sICFG7GFUAA2luY19wZXhwZWN0LnB5AKxbe3PbNrbX58C43RG8laWnXQ7veud7F3FlhNtbdkj
[...]
8Ys0SwQW+DuQhDqXlYSw8fuPPtrNbsrDtjn65LC5Nf8xR4nZvEa2O5y1R0uUHA4/+n+riV9boSgB
AA==
$

2) Get MD5 checksum:

LOCAL $ md5sum pexpect.py
1d9643479e2bf16939fcdf007f4bf9f9 *pexpect.py

3) Add the necessary modules to the script:

import sys
from hashlib import md5 
from cStringIO import StringIO    
from base64 import b64decode 
from gzip import GzipFile

4) Import the module, or create it if it doesn't exist:

try:      
    import pexpect
except ImportError:  
    #Copy and paste the base64 code from step 1
    pexpect_mod = """   
    H4sICFG7GFUAA2luY19wZXhwZWN0LnB5AKxbe3PbNrbX58C43RG8laWnXQ7veud7F3FlhNtbdkj  
    [...]
    8Ys0SwQW+DuQhDqXlYSw8fuPPtrNbsrDtjn65LC5Nf8xR4nZvEa2O5y1R0uUHA4/+n+riV9boSgB    
    AA==   
    """

    #MD5 sum from step 2
    pexpect_mod_md5 = "1d9643479e2bf16939fcdf007f4bf9f9"

    #Decode the module stored in pexpect_mod and load it in a variable
    with GzipFile(mode='r', fileobj=StringIO(b64decode(pexpect_mod))) as pexpect_mod_fd:
        pexpect_mod_data = pexpect_mod_fd.read()

    #Dump the variable into a file
    with open("pexpect.py","w+b") as pexpect_fd:
        pexpect_fd.write(pexpect_mod_data)

    #Double-check pexpect.py is the same file you have in your local machine    
    with open("pexpect.py", 'rb') as pexpect_fd:
        if pexpect_mod_md5 == md5(pexpect_fd.read()).hexdigest():
            #Import the module
            import pexpect
        else:
            #Exit if the file is not identical 
            print("Error creating pexpect.py module. MD5 checksum incorrect. Exiting")
            sys.exit(-1)

And voila, just you run your script in your remote machine, and there you have the module in your current directory:

REMOTE $ ls -1 pexpect.py*
pexpect.py*
pexpect.pyc*

Needless to say you could follow this same approach for any other file you would need to "attach" to your scripts...

Have fun!

Quick Thought I - IPv4 vs TV

psgonza — Wed, 09 Sep 2015 10:10:10 +0200

Watching Mr Robot Episode 1 (I know, I am kind of late, but I have a lot of TV series in my "backlog") I noticed again they used a wrong IPv4 address... Yes, they talk about rootkits, KDE, DDos, etc, etc, but one of the key details of this first episode it's this IP address (not the address itself, but what Malek does with it, if you know what I mean...). How come it's so difficult to remember that only values from 0 and 255 are allowed??? (we could discuss whether they are valid IP addresses for hosts or networks but let's keep it simple)

'218.108.149.373'

And unfortunately this simple error is pretty common in TV shows and movies... The same thing happened several months ago in Blackhat, another "hacking" movie:

'157.257.273.12'

I cannot wait to see what's going to happen when IPv6 finally kicks in :)

Cleaning up my feedly subscriptions

psgonza — Thu, 16 Jul 2015 10:10:10 +0200

(I wrote this post in my way back to the old continent... According to the map, we were flying over Jarenga river, in the mother Russia)

I don't know you, but I have been using RSS clients since, well, foverer I'd say... First Akregator in KDE, then Google Reader (I still wonder why Google had to shut it down) and finally, as most of the people did, I moved to Feedly, and I am pretty happy with it, no complains at all.

A few days ago I was taking a look at my RSS subscriptions and there were more than 400 websites that I have been adding during all these years, so I deciced to clean it up a little bit using Python.

I came up with the script you can see below... It is fairly simple (and it uses only standard library modules) but does the job for me.

In order to use it, you just need to export your feedly subscriptios (you will get an opml file) and execute the script. You will get a new file without the "dead" links ready to be imported.

Usage:

# python3 feedly_checker.py myfeedlyrss.opml

Code:

#!/usr/bin/python3
import os
import sys
from xml.dom import minidom

import http.client
from urllib.parse import urlparse

errors = []

def usage():
    print("Usage: python3 " + sys.argv[0] + " <feedly_export_file>.opml")
    sys.exit(0)


def parse_feed(xmlfile):
    feeds = []
    for _ in range(len(xmlfile)):
        tmp = {}
        if xmlfile[_].getAttribute('type') == "rss":
            tmp[xmlfile[_].getAttribute('text')] = xmlfile[_].getAttribute('xmlUrl')
            feeds.append(tmp)
    return feeds


def check_http_response(url):
    for name, rss in url.items():
        addr = urlparse(rss)
        try:
            conn = http.client.HTTPConnection(addr[1], 80, timeout=3)
            conn.request("HEAD", addr[2])
            res = conn.getresponse()
            if res.status >= 400:
                errors.append(rss)
                print("Name: %s, URL:%s, Error Code:%d" % (name, rss, res.status))
        except http.client.HTTPException:
            errors.append(rss)
        finally:
            conn.close()


def load_xml(filename):
    long_input_file = os.path.realpath(filename)
    try:
        xmldoc = minidom.parse(long_input_file)
        item_res = xmldoc.getElementsByTagName('outline')
    except Exception:
        print("Error parsing xml file")
        sys.exit(1)
    return item_res


def clean_xml(filename, errors):
    xmldoc = minidom.parse(filename)
    doc_root = xmldoc.getElementsByTagName('outline')

    for j in doc_root:
        if j.getAttribute('type') == "rss" and j.getAttribute('xmlUrl') in errors:
            j.parentNode.removeChild(j)
            print("Removing... %s" % (j.getAttribute('xmlUrl')))

    try:
        f = open(filename + "_clean", "w")
        f.write(xmldoc.toxml())
        print("Creating %s_clean file..." % filename)
    except IOError:
        print("Error creating %s_clean file" % filename)
    else:
        f.close()

if len(sys.argv) != 2:
    usage()
else:
    input_file = sys.argv[1]

outlineItems = load_xml(input_file)

results = parse_feed(outlineItems)

print("%d entries found in %s" % (len(results), input_file))

for site in results:
    check_http_response(site)

print("%d wrong entries found" % (len(errors)))

clean_xml(input_file, errors)

sys.exit(0)

It's been a quick excersice (POC?), so there is a lot of room for improvement...

It is not fast, because it is single-thread, and it has to wait for the http connection to timeout.
Any error code above 400 will make the link to be removed... Some error codes 5xx are "temporary server errors", but they will be deteled as well... It'd need some extra work

\\psgonza

Sharing images like a "Pr0"

psgonza — Tue, 14 Jul 2015 13:00:00 +0200

Disclaimer: it's summer, we are under a really long heat wave in Spain, so I didn't feel like moving from the couch (you will understand later :D)

Yesterday evening I was laying on the couch surfing Amazon website using an old laptop (very old) that I barely use, and at some point, I took an screenshot I needed to share with a friend via corporate email... (my idea was to send it to my email account, and this morning, send it from my corporate account)

Anyway, after taking the screenshot I realized:

I was no logged in gmail, dropbox, github, etc , and in order to login, I would need a token from my cellphone (and my smartphone was in a different room!!!)
No USB drives around
The image sharing alternatives that I know of, requires you to create an user and login… Too much for a onetime thing

So I decided to use pastebin to store the image in text format:

Take the screenshot (I used scrot, feel free to use any other tool…) : scrot -d 5 ~/amazon.jpg
Gzip the file: gzip ~/amazon.jpg
Convert to base64: base64 ~/amazon.jpg.gz > ~/amazon.b64
Cat the file and paste the (long) result into pastebin and write down the url

This morning from my corporate laptop:

Download the file from pastebin (save it as amazon.b64)
Set file format UNIX... For instance, using vi: ESC:set ff=unix
Decode it:base64 -d amazon.b64 > amazon.jpg.gz
Unzip it: gunzip amazon.jpg.gz

Et voila, we have the same picture….

I know this doesn't seem very useful (or yes, who knows?) but you might follow the same approach for other things… For instance, I use pretty much the same thing in Python for embedding an old version of pexpect module in my scripts

Later

Issues with Synology 2FA

psgonza — Sat, 11 Jul 2015 00:00:00 +0200

Hi there

After a few months in the shelf, today I reconnected my Synology DS215j, and to my surprise, I wasn't able to login via DiskStation GUI due to a problem with my autenthication:

{% img center /img/synology_ntp_mismatch.jpg 'DSM Error' %}

It was weird because:

a) (you are going to love this one...) "It was working before"
b) I could login to my NAS via SSH with the same user/pass, so it was clearly a problem with the 2 Factor Authethication

Once in SYNOLOGY CLI, I realized the system time was wrong:

{% img center /img/synology_2fa_error.png 'ntp mismatch' %}

As you can (hopefully) see in the picture, the time in the SYNOLOGY CLI (23:58:13) differs from the one in web GUI (23:54) (and the system time in my MBP as well)

So I connected to the NAS with root password (rather sooner than later I would need to "enable" sudo and block root user in SSH) and updated the date by running:

ntpdate -u pool.ntp.org

(If this isn't working, try killing ntpd and run ntpdate -u pool.ntp.org again)

After that, I managed to login using my 2FA code as expected...

Bytes!

PS: First post using github as text editor... Let's see how this goes.

Problems installing IPython Notebook in Cygwin (W7)

psgonza — Wed, 08 Jul 2015 10:41:04 +0200

While trying to "pip install ipython[notebook]" in my Cygwin installation (W7), I got this error due to libzmq:

 Collecting ipython[notebook]

[...]

 Running setup.py install for pyzmq

[...]

gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -ggdb -O2 -pipe -Wimplicit-function-declaration 
-fdebug-prefix-map=/usr/src/ports/python3/python3-3.4.3-1.i686/build=/usr/src/debug/python3-3.4.3-1 
-fdebug-prefix-map=/usr/src/ports/python3/python3-3.4.3-1.i686/src/Python-3.4.3=/usr/src/debug/python3-3.4.3-1 
-DZMQ_USE_POLL=1 -DHAVE_LIBSODIUM=1 -Ibundled/zeromq/include -Ibundled -Ibundled/libsodium/src/libsodium/include 
-Ibundled/libsodium/src/libsodium/include/sodium -I/usr/include/python3.4m -c bundled/zeromq/src/address.cpp 
-o build/temp.cygwin-2.0.4-i686-3.4/bundled/zeromq/src/address.o

gcc: error: spawn: No such file or directory
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-l4nkh9zq/pyzmq/setup.py';
exec(compile(getattr(tokenize,'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" 
install --record /tmp/pip-jicpjqak-record/install-record.txt --single-version-externally-managed --compile" 
failed with error code 1 in /tmp/pip-build-l4nkh9zq/pyzmq

It took me a while to figure it out... In case you were in a similar situation, this is what I did:

1) Install some tools (via apt-cyg or setup.exe)

apt-cyg install libtool automake autoconf

(you will need the usual tools for compiling C/C++ code, such as gcc,g++,make, etc,etc)

2) Export PKG_CONFIG_PATH

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

3) Download and compile libsodium

./configure
make && make install

4) Download and compile libzmq

./autogen.sh
./configure
make && make install

5) Finally, try to install ipython[notebook] again:

pip install ipython[notebook]

It worked for me... Hope it helps you.

Later

From Octopress to Pelican

psgonza — Tue, 07 Jul 2015 23:00:00 +0200

Finally I had some time to move away from Octopress... After trying a couple of Python static web generators I decided to give Pelican a try.

The "migration" was pretty easy. Install it via pip, configure your list of plugins and pages, update your posts (you have to do slightly modifications to your Octopress header posts) and you are good to go... I even managed to keep the same design I already had (thanks to this octopress theme)

So far, it feels good to have an easy bloggin platform in Python, so I totally recommend the change!

Later

Bye 2014 && Hello 2015

psgonza — Sat, 10 Jan 2015 21:41:03 +0200

It is time to say goodbye to 2014... This is how I saw it from my old Samsung Galaxy S3:

{% img center /img/2014byphone.JPG '2014' %}

(50% business/50% leisure)

Happy New Year 2015!!! I'll see you around!

Easy blogging: Octopress, Git and Docker

psgonza — Sun, 02 Nov 2014 10:00:02 +0200

Docker has been probably 2014 revelation (at least in IT world), so I have decided to get my hands on it, and at the same time, try to solve my "problem" with Ruby.

My idea is to "Dockerize" Octopress environment, and run it from a container every time I need to recreate the blog. And also, instead of keeping the blog posts (markdown files) locally, I will store them in a Git repository, and configure Git-hooks so the blog is recreated automatically.

First of all, make sure you have Docker and Git installed in your server.

In order to be able to build the blog, we will need Octopress source code. You can use your current Octopress installation or download it from Github:

$ git clone http://github.com/imathis/octopress.git octopress
$ cd octopress

Once Octopress is downloaded, let's create our Docker container using this Dockerfile:

FROM ubuntu:14.10

RUN apt-get update
RUN apt-get install git ruby1.9.3 ruby-dev build-essential language-pack-en nodejs python -y

ENV LC_ALL en_US.utf8

ADD . /blogSRC
WORKDIR /blogSRC

RUN adduser --uid 1000 --gid 50 deploy
RUN chown -R deploy:staff /blogSRC
USER deploy

RUN gem install bundler
RUN bundle install

ENTRYPOINT ["rake"]

This container will download and install Ruby environment and its dependences (There are some compatibility issues with Python3, so 2.7 has to be installed. See this), so we are able to run rake commands.

Build the container:

$ sudo docker build -t bynario/octopress .
Sending build context to Docker daemon 3.829 MB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.10
 ---> accae238329d
Step 1 : RUN apt-get update
 ---> Running in 5a456e103939
...
...
...
Step 11 : ENTRYPOINT ["rake"]
 ---> Running in c74572f9fb1a
 ---> 94d1b90e8e03
Removing intermediate container c74572f9fb1a
Successfully built 94d1b90e8e03

If it goes fine, you will see your new container by listing the images:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
bynario/octopress   latest              94d1b90e8e03        About an hour ago   531.8 MB
ubuntu              14.10               accae238329d        4 days ago          221.1 MB
$

If you downloaded Octopress from Github in Step 1, then we need to install it. Since we already have our container ready, let's use it:

$ sudo docker run --rm -v /home/docker/octopress/:/blogSRC bynario/octopress install
mkdir -p source
cp -r .themes/classic/source/. source
mkdir -p sass
cp -r .themes/classic/sass/. sass
mkdir -p source/_posts
mkdir -p public
## Copying classic theme into ./source and ./sass
$

(/home/docker/octopress/ is my path to octopress installation folder, change it to fit your installation)

At this point our Octopress is propely installed, and Ruby is running from a Docker container, so there shouldn't be more issues with Ruby and system upgrades.

Now, let's get the posts from our git repository and configure the git-hook. For this I will assume that the posts are already commited and pushed to your git server (Github, BitBicker, etc), and your repositories are working just fine (ssh keys, users, etc..).

cd /home/docker/octopress/source/_posts
git init
git remote add origin git@server.domain:path/repository.git
git pull origin master

(Again, use the path that fits your installation, and update git@server.domain:path/repository.git with your own repository)

Now all the posts are in the _posts folder:

~/octopress/source/_posts$ ls -1
2011-01-01-tips-instalar-vmware-en-opensuse-11.markdown
...
...
2014-10-18-new-look-same-engine.markdown
2014-10-18-python-port-locker-slash-unlocker.markdown
~/octopress/source/_posts$

So we could generate the blog and check that it works. Let's run our docker container:

$ sudo docker run --rm -v /home/docker/octopress/:/blogSRC bynario/octopress generate

docker run --rm: run Docker and automatically clean up the container and remove the file system when the container exits.
-v /home/docker/octopress/:/blogSRC: Volume sharing. Mounting /home/docker/octopress/ in the local machine as /blogSRC in the container.
bynario/octopress: Image name and tag.
generate: Rake command to be executed by the container.

And finally let's create our git-hook. There are (at least) two different ways of working:

Webhooks: The repository will communicate with a web server whenever the repository is pushed to.
Client-Side Hooks: Local scripts that are called when certain events occur.

I chose the latter. So I configured a post-merge hook, that will be executed every time the content of my _post directory is updated after running a git pull:

sudo docker run --rm -v /home/docker/octopress/:/blogSRC bynario/octopress generate

Don't forget the execution rights: chmod +x /home/docker/octopress/source/_posts/.git/hooks/post-merge

Now, it is up to you how to call the git-hook. For example, you could run it manually every time you post something new or schedule it every night in crontab:

* 00 * * *  cd /home/docker/octopress/source/_posts && /usr/bin/git pull origin master

Happy blogging!

Python port locker/unlocker

psgonza — Sat, 18 Oct 2014 17:58:30 +0200

If you run you own server (VPS, cloud instances, bare metal, no matter what), I guess you check your log files almost on daily basis, and probably you already run any the multiple available tools (the list is pretty long, but in my case I use logwatch) that helps you out to keep an eye in what is going on in your machine.

There is one thing in particular that drives me crazy. As soon as you expose a machine to the internet, people will be constantly trying to get SSH access:

{% img center /img/logwatch_print.JPG 'logwatch' %}

Obviously, there are tools that will make your life easier (like fail2ban, totally recommended) and some security tweaks (just Google for "linux server hardening") that you should be doing if you manage your own server, but even with all those things in place, I really don't like people trying to access to my server 24/7... Call me crazy, if you like.

A simple solution would have been just to change the SSH port to a something different, like 22222, and this might work for you. Unfortunately, I spend most of the time behind a proxy server, and I can only access the most common ports (the same very ones that are scanned on regular basis), so I needed to keep ssh service running in port 22.

Last year I coded a small bash script (just for myself) that allowed me to open and close SSH port on demand. It is been working fine since then, and I don't get annoyed by the ssh auth errors in logwath anymore. Now I migrated it to Python, and I thought it could be useful for anyone else, so here it is...

The script is not big deal, it will open/close certain ports that I preconfigured by reading a text file. Nothing new under the sun.

The only "particularity" of this script is that, instead of creating a service (webservice??) that allowed me to send the desired ssh state to the server (this would add another "attack vector"), I decided the script would pull the data from another location. I decided to use a (public) file in my dropbox folder.

If you want it to give it a try, these are the steps:

1 - Create a file in your dropbox folder and get the public link. Let's say that you get something like this:

https://www.dropbox.com/s/XXXXXXXXXXXXXXXX/file

2 - Add to this file any of the options you defined in the script. ie:

open ssh
close vpn
open  httpd

3 - Upload this script to your server (github link available in the code snippet header):

#!/usr/bin/env python
#===============================================================================
#
#          FILE:  portLocker.py
#         USAGE:  python portLocker.py
#   DESCRIPTION:  port blocker/unblocker via dropbox file
#        AUTHOR:  http://bynario.com
#       CREATED:  05/08/2014 17:37:23 CEST
#===============================================================================
import time,sys,os,subprocess

dbFile="https://www.dropbox.com/s/XXXXXXXXXXXXXXXX/file"
logFile="/var/log/locker_log.log"
htmlFile="/usr/share/nginx/www/sshd.html"
servicesL={'ssh':'22','vpn':'1723','httpd':'80'}
actionsL=('open','close')

def createWWW(filename):
    try:
        with open(filename, 'w') as f: f.write(getTime())
    except Exception as e:
        print2Log(logFile,"Error opening "+filename+": "+str(e))

def print2Log(filename,text):
    try:
        with open(filename, 'w') as f:
            f.write(getTime() + " - " + text+"\n")
    except Exception as e:
        print2Log(logFile,"Error opening "+filename+": "+str(e))

def print2WWW(filename,service,status):
    try:
        with open(filename, 'a') as f:
            f.write(" - Service: " + service + " - Status: " + status)
    except Exception as e:
        print2Log(logFile,"Error opening "+filename+": "+str(e))

def getTime():
    return time.strftime("%Y-%m-%d %H:%M")

def runCommand(command,retVal=None):
    try:
        p = subprocess.Popen(command.split(), stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=False)
        (output, err) = p.communicate()
        if retVal: return output
        else: return None
    except Exception as e:
        print2Log(logFile,"Error executing "+str(command)+": "+str(e))

def checkDBFile():
    action="/usr/bin/curl -s -L " + dbFile
    command=runCommand(action,retVal=True)
    return command

def runIptables(service,action):
    print getTime() + " Running iptables... Service: " + service + " Action:" + action
    check_command="/sbin/iptables -C INPUT -p tcp -m tcp --dport " + servicesL[service] +" -j DROP"
    try:
        check_output = subprocess.check_call(check_command.split(),stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as e:
        if action == "close": 
            command="/sbin/iptables -A INPUT -p tcp --dport " + servicesL[service] + " -j DROP"
        else: 
            return True
    else:
        if action == "open":
            command="/sbin/iptables -D INPUT -p tcp --dport " + servicesL[service] + " -j DROP"
        else: 
            return

    runCommand(command)

if __name__ == "__main__":
    createWWW(htmlFile)

    for line in checkDBFile().splitlines():
        if not line: continue
        if len(line.split()) != 2:
            print2Log(logFile,"Error in dropbox file: wrong format")
            sys.exit(1)
        else:
            act = line.split()[0]
            serv = line.split()[1]

            if servicesL.has_key(serv) and act in actionsL:
                runIptables(serv,act)
                print2WWW(htmlFile,serv,act)
            else:
                print
                print2Log(logFile,"Service or Action found in file not valid... Skipping: " + act + "-" + serv)

4 - Now check these variables in the script:

dbFile="https://www.dropbox.com/s/XXXXXXXXXXXXXXXX/file"
logFile="/var/log/locker_log.log"
htmlFile="/usr/share/nginx/www/sshd.html"
servicesL={'ssh':'22','vpn':'1723','httpd':'80'}

Adapt the script to fit your environment:

Replace dbFile by the URL you got from Dropbox
Set the path you need for the logFile
The script will write the status of the services to my web server, so I can check the status easily. Set your path in htmlFile.
Define the list of services you want to manage in servicesL. This has to match with the info you define in the dropbox file.

5 - Create an entry in crontab with the periodicity you need. In my case, opening/closing ssh port is not critical, so, as I don't want Dropbox people to get angry at me, I set it up to 5 minutes :)

 */5 * * * * python /path/to/portLocker.py &>> /var/log/locker_log.log

Now all you need to do is modify your dropbox file, and after a few minutes, check if you get the result you expected ;)

There is one main improvement that could be done, and it is adding some kind of signature to the file in order to ensure nobody is messing with us, but I have pretty good reasons not to do it (at the moment):

I update the dropbox file from mobile devices every now and then, and adding a signature to the file would make this way too complex.
This file is stored in my dropbox folder, so theoretically, nobody but me is able to modify it.
The security of the server does not rely on this script. This is not meant to be "security through obscurity".

If you are after something more professional, I recommend you to check Latch.

And that's all folks... It is working pretty well for me, and I hope it works for you...

See you around

New look. Same engine

psgonza — Sat, 18 Oct 2014 16:35:28 +0200

Dear reader, if you have ever been in a (long) business trip, you would problably agree with me on this:

"At the end, long business trips are pretty boring" (ok, ok, it depends on the place)

I have been in Amman (Jordan) for the past few weeks. I have seeing what could be seen around (not that much IMO), but since now I have some "spare" time, I have decided to take another look at Octopress. (I still think I would be better off without Ruby though :-P)

So, as you can see, I have modified the CSS of the site to make it cleaner and lighter, because I never really like dark styles.

Actually, this is how bynario.com looked like in 2005 (I found this in archive.org):

{% img center /img/bynarioSS-feb2006.JPG '2006' %}

And this is in 2011 (I hosted this site in blogger for a few months):

{% img center /img/bynarioSS-oct2011.JPG '2011' %}

And finally, since June 2012, I have been running Octopress with just some small modifications over the default theme:

{% img center /img/bynarioSS-oct2014.JPG '2014' %}

Now I just need to stop procrastinating and post more often... Easier said than done!

Bye

Blog engines comparation

psgonza — Fri, 17 Oct 2014 16:11:51 +0200

I don't like Ruby, at all... Maybe because I never really "used it" so didn't have the opportunity to know its guts, or maybe because it just sucks... If you ask me, it is probably the latter :P

As you might know, Octopress (which is a great blog engine, kudos to Brandon Mathis) is written in Ruby, so every now and then I have to deal with rvm, and all the ruby paraphernaliam when an upgrade breaks it in my VM.

Working in an environment you are not familiar with is not easy and every time it breaks, I have to spend some time trouble-shooting it, so I have decided to take a look at other available blog engines, compare them with Octopress, and migrate to a more "friendly" environment (hopefully)

So I compared these three alternatives: Octopress, Tinkerer and Hugo

Below you can see some facts and personal opinions about the three engines:

Octopress:
- Written in Ruby
- Based on Jekyll
- Ready to use out of the box. (version 3.0 in progress)
- Installation pretty well documented, but requieres setting up ruby environment
- Not that really fast... at all
- Very mature (it's been around for 3 o 4 years)
- Add-on plugins
- 8380 stars in GitHub

{% img center /img/octopress_Capture.JPG octopress %}

Tinkerer:
- Written in Python
- Based on Sphinx
- No markdown
- It needs a few dependences (like python-dev libxml2-dev libxslt-dev lib32z1-dev libz-dev)
- Fast
- 156 stars in GitHub

{% img center /img/tinker_Capture.JPG tinkerer %}

Hugo:
- Written Golang
- Using its own generation engine
- Highly customizable. Barely usable right after installation
- Run in multiple OS (including Windows)
- Really easy to install. Binary available for several OS
- Oh boy this is FAST.
- 2239 stars in GitHub

{% img center /img/hugo_Capture.JPG hugo %}

Besides installing and playing around with them, I decided to run a small benchmark, just to compare the performance of the different engines. Basically this is what I did:

1 - Generate 500 random posts:

$ time for i in `seq 1 500`; do rake new_post["post $RANDOM $RANDOM"];done
Creating new post: source/_posts/2014-10-15-post-25028-3011.markdown
mkdir -p source/_posts
Creating new post: source/_posts/2014-10-15-post-16477-16246.markdown
mkdir -p source/_posts
[...]
Creating new post: source/_posts/2014-10-15-post-1586-30872.markdown
mkdir -p source/_posts
Creating new post: source/_posts/2014-10-15-post-680-27529.markdown

real    7m35.910s
user    6m53.838s
sys     0m38.886s

$ time for i in `seq 1 500`; do tinker --post "post $RANDOM";done
New post created as '/root/bynario/2014/10/15/post_15124.rst'
New post created as '/root/bynario/2014/10/15/post_19047.rst'
[...]
New post created as '/root/bynario/2014/10/15/post_14905.rst'
New post created as '/root/bynario/2014/10/15/post_10370.rst'
New post created as '/root/bynario/2014/10/15/post_4049.rst'

real    2m18.046s
user    1m55.815s
sys     0m18.737s

$ time for i in `seq 1 500`; do hugo new post/post_${RANDOM}_${RANDOM}.md;done
/root/hugoblog/content/post/post_24649_29646.md created
/root/hugoblog/content/post/post_24308_8060.md created
/root/hugoblog/content/post/post_8311_6440.md created
[...]
/root/hugoblog/content/post/post_12973_1855.md created
/root/hugoblog/content/post/post_23049_13857.md created
/root/hugoblog/content/post/post_26136_21085.md created

real    0m15.123s
user    0m7.424s
sys     0m5.688s

2 - Add some text (no markdown, just 50 lines of plain text) to each post in order not to generate (almost) empty html pages

3 - Build the whole site

$ time rake generate
## Generating Site with Jekyll
identical source/stylesheets/screen.css
Configuration file: /root/octopress/_config.yml
            Source: source
       Destination: public
      Generating... done.
 Auto-regeneration: disabled. Use --watch to enable.

real    2m45.143s
user    2m41.834s
sys     0m3.172s

$ time tinker --build
Making output directory...
Running Sphinx v1.2.3
loading pickled environment... not yet created
building [html]: targets for 502 source files that are out of date
updating environment: 502 added, 0 changed, 0 removed
reading sources... [100%] master
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] master
writing additional files... search
copying static files... WARNING: favicon file 'tinkerer.ico' does not exist
done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 1 warning.

real    0m59.897s
user    0m57.792s
sys     0m1.328s

$ time hugo  --theme=liquorice --buildDrafts
498 of 498 drafts rendered
0 future content
498 pages created
0 categories created
0 tags created
in 2098 ms

real    0m2.136s
user    0m1.860s
sys     0m0.252s

Let's put the results in a table... I think figures speak for themselves:

	Octopress	Tinkerer	Hugo
Page generation	7m35.910s	2m18.046s	15.123s
Site generation	2m45.143s	59.87s	2.136s

(The result may vary running the test in bare metal instead of in a -crappy- VM, but still they are very descriptives)

Based on the results:

Ok, so Octopress is slow, and old... So what? It works really good out of the box, speed would be a problem if you post several times per day, but in my case, so it is not really an issue. For me the only thing that makes me think about changing the blog engine is Ruby... As simple as that
Tinkerer looks like a neat product, and I will problably take another look at it, but since it doesn't allow markdown, I am not so sure if I feel like migrating the post to a different format... Still, it is based in Python, and it's faster than Octopress, so I will give it a try...
Then should we move to Hugo blindfolded?? I don't think so... Not just yet. It is true that it is way faster, but I found it pretty complex for someone that doesn't really want to build the blog "almost" from scratch. Nevertheless, it looks really promising, it is surprisingly fast, and run in multiple OS (even from Windows)... So it might be a perfect alternative (at least for me) as soon as it is a little bit more mature.

Tricky decision...Should I stay or should I go?? :)

Bye!

Python: Parsing XML files

psgonza — Tue, 05 Aug 2014 13:14:00 +0200

{% img center /img/python-xml.jpg 'xml' %}

I would like to continue with another small python example. As in the previous post,this is more like a "note to self" thing than a educational post (this is not stackoverflow), but anyway, I guess it could be handy for someone(it has been for me...)

This time, I have been testing two xml submodules availables in Python 2.7 XML package:

xml.dom: the DOM API definition
xml.etree.ElementTree: the ElementTree API, a simple and lightweight XML processor

Both of them are pretty easy to use, and I haven't found any real difference in time execution (at least for basic filters) between both submodules.

I will use this xml file as an example:

<?xml version="1.0"?>
<data>
    <node>
        <attribute>Front End</attribute>
        <res_ids>100</res_ids>
        <nEName>BALDR</nEName>
        <ipList>192.168.0.5</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Web</attribute>
        <res_ids>12</res_ids>
        <nEName>THOR</nEName>
        <ipList>192.168.0.20</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Storage</attribute>
        <res_ids>200</res_ids>
        <nEName>VALI</nEName>
        <ipList>192.168.0.10</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>DB</attribute>
        <res_ids>230</res_ids>
        <nEName>LOKI</nEName>
        <ipList>192.168.0.110</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Fedora</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Backup</attribute>
        <res_ids>300</res_ids>
        <nEName>OTHER</nEName>
        <ipList>192.168.0.103</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Debian</so>
        <kernel>3.0.93</kernel>
    </node>
</data>

In the tree, we can see that we have one "data" object containing several "nodes", each one with 7 "attributes".

Now let's extract the nodes and all the attributes using xml.dom:

#!/usr/bin/env python
from xml.dom import minidom

def parse_file(filename):
    xmldoc = minidom.parse(filename)
    for node in xmldoc.getElementsByTagName('node'):
        print str(node.getElementsByTagName("attribute")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("res_ids")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("nEName")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("ipList")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("ipv6List")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("so")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("kernel")[0].firstChild.nodeValue)

parse_file("./fakexml.xml")

And now, the same thing using xml.etree.ElementTree:

#!/usr/bin/env python
from xml.etree import ElementTree as ET

def parse_file(filename):
    tree = ET.ElementTree(file=filename)
    attrList = [ entry for entry in tree.findall('./node') ]
    for e in attrList:
        print e.find('attribute').text,
        print e.find('res_ids').text,
        print e.find('nEName').text,
        print e.find('ipList').text,
        print e.find('ipv6List').text,
        print e.find('so').text,
        print e.find('kernel').text

parse_file("./fakexml.xml")

The output from any of the two options would be the same one:

$> python print_table_from_xml_dom.py
Front End 100 BALDR 192.168.0.5 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Web 12 THOR 192.168.0.20 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Storage 200 VALI 192.168.0.10 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
DB 230 LOKI 192.168.0.110 fe80::1:f6f1:fe01:12 Fedora 3.0.93
Backup 300 OTHER 192.168.0.103 fe80::1:f6f1:fe01:12 Debian 3.0.93


$> python print_table_from_xml_element.py
Front End 100 BALDR 192.168.0.5 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Web 12 THOR 192.168.0.20 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Storage 200 VALI 192.168.0.10 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
DB 230 LOKI 192.168.0.110 fe80::1:f6f1:fe01:12 Fedora 3.0.93
Backup 300 OTHER 192.168.0.103 fe80::1:f6f1:fe01:12 Debian 3.0.93

As I mentioned before, for this small exercise, there is no difference in terms of time of execution between both alternatives. I created a "slightly bigger" xml file (100x bigger), and the times were still pretty much the same, so I would say it is not a matter of the size, but the complexity of the filters.

To wrap up, let's put this code together with the one in the previous post, where we created a table from the data in a dictionary.

I use ElementTree instead of xml.dom, but both of them would work. Here's the code:

#!/usr/bin/env python
import sys
from collections import defaultdict
from xml.etree import ElementTree as ET

if sys.argv[1:]:
    inputF = sys.argv[1]
else:
    print "ERROR. Execution failed: missing file"
    sys.exit(1)

total = defaultdict(list)

# Column width
AttrColLen=25
ValueColLen=20
colwidth1="{0:<"+str(AttrColLen)+"}"
colwidth2="{0:<"+str(ValueColLen)+"}"


# Printer function
def print_table(data):
    for key, values in sorted(total.items()):
        print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"
        print  "| " + colwidth1.format(key) + "|",
        for i in xrange(len(values)):
            print colwidth2.format(values[i])+"|",
        print ""
    print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"

# Parse the xml, and add the items to the dictionary
def parse_file(filename):
    colwidth1="{0:<30}"
    colwidth2="{0:<25}"

    tree = ET.ElementTree(file=filename)
    attrList = [ entry for entry in tree.findall('./node') ]
    for e in attrList:
        total["Attribute"].append(e.find('attribute').text)
        total["RES_ID"].append(e.find('res_ids').text)
        total["networkElementName"].append(e.find('nEName').text)
        total["IPv4"].append(e.find('ipList').text)
        total["IPv6"].append(e.find('ipv6List').text)
        total["SO"].append(e.find('so').text)
        total["Kernel"].append(e.find('kernel').text)


parse_file(inputF)
print_table(total)

And here is the result:

$> python print_table_from_dict.py fakexml.xml
|----------------------------------------------------------------------------------------------------------------------------------------|
| Attribute                | Front End           | Web                 | Storage             | DB                  | Backup              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| IPv4                     | 192.168.0.5         | 192.168.0.20        | 192.168.0.10        | 192.168.0.110       | 192.168.0.103       |
|----------------------------------------------------------------------------------------------------------------------------------------|
| IPv6                     | fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12|
|----------------------------------------------------------------------------------------------------------------------------------------|
| Kernel                   | 3.0.93              | 3.0.93              | 3.0.93              | 3.0.93              | 3.0.93              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| RES_ID                   | 100                 | 12                  | 200                 | 230                 | 300                 |
|----------------------------------------------------------------------------------------------------------------------------------------|
| SO                       | Ubuntu              | Ubuntu              | Ubuntu              | Fedora              | Debian              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| networkElementName       | BALDR               | THOR                | VALI                | LOKI                | OTHER               |
|----------------------------------------------------------------------------------------------------------------------------------------|

;)

Python: print table from dictionary of lists

psgonza — Thu, 31 Jul 2014 13:14:00 +0200

{% img center /img/python-logo-inkscape.jpg 'python' %}

Hello

I have been trying to find an easy way to create ascii tables in Python using data from a dictonary of lists. I couldn't find anything that suited me, so I came up with this small piece of code

{% codeblock %} def print_table(data): for key, values in sorted(total.items()): print "|" + AttrColLen"-" + ((ValueColLen+2)len(values))"-" + "-|" print "| " + colwidth1.format(key) + "|", for i in xrange(len(values)): print colwidth2.format(values[i])+"|", print "" print "|" + AttrColLen"-" + ((ValueColLen+2)len(values))"-" + "-|"

It's pretty simple, and does the job. Let me show it to you with an example

#!/usr/bin/env python
from collections import defaultdict

total = defaultdict(list)

# Column width
AttrColLen=25
ValueColLen=15
colwidth1="{0:<"+str(AttrColLen)+"}"
colwidth2="{0:<"+str(ValueColLen)+"}"

# Fake data 
attribute=['Front End','Web','Storage']
res_ids=['100','12','200']
nEName=['BALDR','THOR','VALI']
ipList=['192.168.0.5','192.168.0.20','192.168.0.10']

# Printer
def print_table(data):
    for key, values in sorted(total.items()):
        print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"
        print  "| " + colwidth1.format(key) + "|",
        for i in xrange(len(values)):
            print colwidth2.format(values[i])+"|",
        print ""
    print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"



# Populating the dictionary
for iter1 in attribute:
    total["Attribute"].append(iter1)
for iter2 in res_ids:
    total["RES_ID"].append(iter2)
for iter3 in nEName:
    total["networkElementName"].append(iter3)
for iter4 in ipList:
    total["IPv4"].append(iter4) 

# Let's print the dictionary
print total

Obviously there are other (better!!) ways to print a dictionary (like pprint),anyway I just wanted to show you how the dictionary looks like

defaultdict(<type 'list'>, {'Attribute': ['Front End', 'Web', 'Storage'],
'RES_ID': ['100', '12', '200'], 
'networkElementName': ['BALDR', 'THOR', 'VALI'], 
'IPv4': ['192.168.0.5', '192.168.0.20', '192.168.0.10']})

Now let's replace:

# Let's print the dictionary
print total

By:

# Let's print the dictionary
print_table(total)

This is the result:

|-----------------------------------------------------------------------------|
| Attribute                | Front End      | Web            | Storage        |
|-----------------------------------------------------------------------------|
| IPv4                     | 192.168.0.5    | 192.168.0.20   | 192.168.0.10   |
|-----------------------------------------------------------------------------|
| RES_ID                   | 100            | 12             | 200            |
|-----------------------------------------------------------------------------|
| networkElementName       | BALDR          | THOR           | VALI           |
|-----------------------------------------------------------------------------|

It looks pretty nice, and the code is really simple...

I hope it helps

Byte!

Network manager for windows 7 using cmd.exe

psgonza — Sun, 16 Feb 2014 13:14:00 +0100

{% img center /img/cmd_exe.JPG 'cmd.exe' %}

Updated version of the script. Now it allows you to enable/disable the network interfaces

Hi there

Lately I have been changing the network properties (ip addresses, netmask, etc etc) in my laptop (W7 Enterprise 64b) several times per day, and to be honest I am growing tired of it, so I created a small bat script in order to make my life easier

The script is pretty simple... It might be handy for you as well, so here you are:

@ECHO OFF
SET OPTION=

goto :MENU

:MENU
cls
Echo #### CLI NETWORK MANAGEMENT ####
Echo.  
Echo ****** LAN ******
Echo 1) LAN - DHCP
Echo 2) LAN - STATIC IP:192.168.2.10/24 - GW:192.168.2.99 - DNS:8.8.8.8
Echo 3) LAN - DEFINE NEW STATIC IP
Echo.
Echo ****** WIRELESS LAN ******
Echo 4) WLAN - DHCP
Echo 5) WLAN - STATIC IP:192.168.2.20/24 - GW:192.168.2.99 - DNS:8.8.8.8
Echo 6) WLAN - DEFINE NEW STATIC IP
Echo.
Echo ****** INTERFACES ******
Echo 7) SHOW INTERFACES STATUS
Echo 8) ENABLE LAN
Echo 9) DISABLE LAN
Echo 10) ENABLE WLAN
Echo 11) DISABLE WLAN
Echo.
Echo 0) Exit
Echo. 
set /p OPTION=Option: 
Echo. 

IF "%OPTION%"=="0" call :END
IF "%OPTION%"=="1" call :Opt1 "Local Area Connection"
IF "%OPTION%"=="2" call :Opt2 "Local Area Connection" "192.168.2.10" "255.255.255.0" "192.168.2.99" "8.8.8.8"
IF "%OPTION%"=="3" call :Opt3 "Local Area Connection"
IF "%OPTION%"=="4" call :Opt1 "Wireless Network Connection"
IF "%OPTION%"=="5" call :Opt2 "Wireless Network Connection" "192.168.2.20" "255.255.255.0" "192.168.2.99" "8.8.8.8"
IF "%OPTION%"=="6" call :Opt3 "Wireless Network Connection"
IF "%OPTION%"=="7" call :Opt4 
IF "%OPTION%"=="8" call :Opt41 "Local Area Connection" "ENABLED"
IF "%OPTION%"=="9" call :Opt41 "Local Area Connection" "DISABLED"
IF "%OPTION%"=="10" call :Opt41 "Wireless Network Connection" "ENABLED"
IF "%OPTION%"=="11" call :Opt41 "Wireless Network Connection" "DISABLED"


goto :MENU

:Opt1
REM DHCP
REM "%~1" - Interface Name
cls
Echo Configuring %~1 as DHCP...
Echo. 
Echo Configuring DHCP for IP
netsh interface ip set address "%~1" dhcp
Echo Configuring DHCP for dns
netsh interface ip set dns "%~1" dhcp
Echo Configuring DHCP for wins
netsh int ip set wins name="%~1" dhcp
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
Echo. 
pause
goto :MENU


:Opt2
REM STATIC IP
REM "%~1" - Interface Name
REM "%~2" - IP
REM "%~3" - NETMASK
REM "%~4" - GW
REM "%~5" - DNS
cls
Echo Configuring static IP address for %~1
Echo =============================================
Echo.
Echo Configuring IP %~2 %~3 %~4
netsh interface ip set address name="%~1" static "%~2" "%~3" "%~4"
Echo Configuring DNS %~5
netsh interface ip set dns name="%~1" static "%~5"
netsh interface ip add dns name="%~1" 8.8.4.4
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
pause
goto :MENU

:Opt3
REM STATIC IP - NEW
REM "%~1" - Interface Name
cls
set /p IP=IP:
set /p NET=NETMASK:
set /p GW=GATEWAY:
set /p DNS=DNS:
Echo Configuring new static IP address for %~1
Echo =============================================
Echo.
Echo Configuring IP %IP% %NET% %GW%
netsh interface ip set address name="%~1" static "%IP%" "%NET%" "%GW%"
Echo Configuring DNS %DNS%
netsh interface ip set dns name="%~1" static "%DNS%" 
Echo Configuring secondary DNS 8.8.4.4
netsh interface ip add dns name="%~1" 8.8.4.4
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
pause
goto :MENU

:Opt4
REM INTERFACES
REM "%~1" - Interface Name
REM "%~2" - status
cls
Echo ****** INTERFACES ******
Echo. 
Echo Checking network interface status...
Echo.
netsh interface  show interface | find /i "connection"
Echo.
pause
goto :MENU

:Opt41
cls
Echo ****** INTERFACES ******
Echo. 
Echo Setting %~1 status to ... %~2 
netsh interface set interface name="%~1" admin="%~2" 
goto :Opt4


:END
exit

The script needs to be executed as administrator...

Online again

psgonza — Mon, 27 Jan 2014 13:14:00 +0100

It's been a while.... but better late than never, isn't it? I am back online (I never left, but this site didn't come back after a VPS migration)...

Not a single post in 2013... "Working, working and working" that would sum up the whole last year.

These are my thoughts about a few things that happened in 2013:

RaspberryPi:

I got mine (pre-ordered) in the fall of 2011... Well actually I paid it then, but it was delivered in Jun 2012. In 2013 I bought another one to use it as a media center with raspbmc. What a piece of HW for only 37 euros.

Python:

I know I know, Python is OLD, but believe or not, I never used it before until 2013. I am not a developer, but Bash and Perl gave me pretty much everything I needed... Until I started to play with Python, it's easy, fast, and there are modules for almost everything you can think of.

Ansible:

According to them, "a radically simple IT orchestration solution", and you bet it is. Give it a chance if you can. Way easier than Puppet/Chef/Fabric...

I think the number of users in Google+ is ramping up mainly because it's mandatory if you want to use some Google products. Unfortunatelly, it didn't reach my expecations. There are some interesing stuff, but I am kind of bored of seeing animated gifs.

New family member:

After a few years using "old" computers (and my company laptop), I decided to buy a Lenovo x230, together with a 256GB Samsung SSD 840 PRO. To put is simple, it is F.A.S.T. As usually, GNU/Debian is the OS of choice. Still, some days I have second thoughs about it. Should I have bought a MBA 11" or this one?? I haven't reach any conclusion yet :)

Bye bye Google Reader

At the end, it wasn't a big deal. I had been a G.Reader power user for years, but due to my limited time, I started to check my feeds in the tablet (Nexus 7 2012) instead than in the laptop, and moving to Feedly was a smooth migration. You know what they say... the king is dead, long live the king.

Iceland and again to the US:

I did spent the whole year working long hours and weekends, but at least, I could take a few days off (in two different periods) and visited Iceland and some cities in the US East Coast. Iceland is amazing, it is like Jurasic Park (geologically speaking). You should totally go.

{% img center /img//DSC_0597.jpg 'Iceland' %}

And again, back to the US... We had a really good time, as usual.

{% img center /img//DSC_0600.NEF.jpg 'NYC' %}

{% img center /img//DSC_1057.NEF.jpg 'chicago' %}

I will try to write a few post this year, but I can't promise anything

//Bye

Monitoring remote systems: Using GTalk notifications

psgonza — Sun, 05 Aug 2012 13:52:00 +0200

Since a few weeks ago I am using a Raspberry Pi as bittorrent client instead of my old, big and noisy Pentium III 1Ghz

{% img center /img/DSC_0321_sm.jpg 'admin' %}

I am pretty happy with the change (besides a few issues with the USB/Network controller). I managed to get rid of a big PC case, there are no fans spinning anymore (noiseless), and the power consumption is unbeatable, and I only paid around $40. Win-Win situation, isn't it?.

But this post is not about my rasPi (ok, maybe just a little bit), but about how to received XMMP notification from a remote system. In this case, I was interested in sending a brief report of my VPS status (CPU,MEM,HD, etc) every 12 hours to my Gtalk account, so I can see how it is doing.

Most of the alternatives are based on sending emails, but I wasn't very keen on setting up a MTA only to send a few reports every X hours, so I decided to use XMMP notifications instead.

Googling a little bit I found this:

apt-cache show sendxmpp
Package: sendxmpp
Priority: optional
Section: net
Installed-Size: 88
Maintainer: Guus Sliepen <guus@debian.org>
Architecture: all
Version: 1.20-1
Depends: perl, libnet-xmpp-perl
Filename: pool/main/s/sendxmpp/sendxmpp_1.20-1_all.deb
Size: 15026
MD5sum: 2a3fcdc995442ebd1f7d8a336fc88164
SHA1: 90dd5cc21c9b97872dec5a55ce9d55ea7d728d65
SHA256: eb8a2fd09ffc65c04cc54a8a718faf550b5864a3135a9d045d2657cd43ed0449
Description: commandline XMPP (jabber) utility
 sendxmpp is a perl script to send XMPP (jabber) messages, similar to what
 mail(1) does for mail. XMPP is an open, non-proprietary protocol for instant
 messaging. See www.jabber.org for more information.
Tag: implemented-in::perl, interface::commandline, protocol::jabber, role::program, scope::utility

Just what I needed.

PROs: - Easy to use and integrate in simple scripts. - Perl script available in Debian repositories. - Lightweight. There are no daemons running. The script is executed whenever is needed. - Secure. Using SSL.

CONS: - Only available to GTalk users (obviously). - Limited size of the message. (just because of the limited size of the screen of your mobile device) - In order to avoid SPAM, the account where you are sending the message from, has to be in the contact list of the receiver.

As, it was working like a charm in my Raspberry Pi, I decided to do the same in my VPS. Let me describe in a few steps how to configure it:

Installation? Debian's way.

 apt-get install sendxmpp
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  libauthen-sasl-perl libdigest-sha1-perl libio-socket-ssl-perl libnet-libidn-perl libnet-ssleay-perl libnet-xmpp-perl
  libxml-stream-perl
Suggested packages:
  libdigest-hmac-perl libgssapi-perl libio-socket-inet6-perl libnet-dns-perl
The following NEW packages will be installed:
  libauthen-sasl-perl libdigest-sha1-perl libio-socket-ssl-perl libnet-libidn-perl libnet-ssleay-perl libnet-xmpp-perl
  libxml-stream-perl sendxmpp
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 581 kB of archives.
After this operation, 2,543 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y

Pretty easy configuration. Under your home directory, create a .sendxmpprc file

# cat .sendxmpprc
<your_email>@gmail.com <your_password>

Let's test it. First sending a message from the standard input:

echo "hello there" | /usr/bin/sendxmpp  -t -u <your_email> -o gmail.com <recipient Google ID>

Notification received:

{% img center /img/gtalk1.jpg %}

And now, sending the text from a file:

echo "this is a message send from a text file" > text.txt
/usr/bin/sendxmpp -t -u <your_email> -o gmail.com <recipient Google ID> -m text.txt

Notification received:

{% img center /img/gtalk2.jpg %}

Working!

Integrating it in a simple bash script:

#!/bin/bash

REPORT="/tmp/.report"

[  -f $REPORT ] && rm -f $REPORT
echo  "VPS report at `date`:" >> $REPORT
echo  "Machine stats : `uptime`" >> $REPORT
echo  "Memory stats : `free -m`" >> $REPORT
echo  "Hard Disk Space: `df -h`" >> $REPORT
echo "" >> $REPORT


#Sending the report to GTALK
/usr/bin/sendxmpp -t -u <your_email> -o gmail.com <recipient Google ID> -m $REPORT

And setting up the task in cron:

crontab -e
57 17 * * * sh /scripts/send_report.sh

The notification received in my Gtalk client:

{% img center /img/gtalk3.jpg %}

This is just an example, but I am pretty sure you could find a lot of applications to this basic notification system

Enjoy!

Cacti monitoring over SSH

psgonza — Sat, 23 Jun 2012 09:26:00 +0200

Recently I had to configure Cacti in order to monitor several blades in a cluster (nothing fancy, just CPU load, memory usage, load average and disk space) but this time NET-SNMP wasn't an option. I knew that I could use server scripts to access the remote blades using SSH, so after googling a while, I found this proejct from the Percona guys:

Percona Monitoring Plugins for Cacti

What can I say... Amazing. Everything working in a few minutes.

Although all the information about Cacti and Percona plugins can be found in their web sites, I decided to put it all together and take a few screenshots, so if you happen to do something similar, this is what I did:

Let's assume that you have a fresh a Debian box (In my case, I created a Debian virtual machine, running testing for this)

1) Install cacti (and its dependences) using aptitude

aptitude install cacti
The following NEW packages will be installed:
  apache2-mpm-prefork{a} apache2-utils{a} apache2.2-bin{a}
  apache2.2-common{a} cacti dbconfig-common{a} libaio1{a}
  libapache2-mod-php5{a} libaprutil1-ldap{a} libcairo2{a} libdatrie1{a}
  libdbd-mysql-perl{a} libdbi-perl{a} libdbi0{a} libfontenc1{a}
  libhtml-template-perl{a} libmysqlclient16{a} libnet-daemon-perl{a}
  libonig2{a} libpango1.0-0{a} libpango1.0-common{a} libphp-adodb{a}
  libpixman-1-0{a} libplrpc-perl{a} libpng12-0{a} libqdbm14{a} librrd4{a}
  libthai-data{a} libthai0{a} libx11-6{a} libx11-data{a} libxau6{a}
  libxcb-render-util0{a} libxcb-render0{a} libxcb1{a} libxdmcp6{a}
  libxfont1{a} libxft2{a} libxrender1{a} mysql-client-5.1{a}
  mysql-client-5.5{ab} mysql-common{a} mysql-server{a} mysql-server-5.5{ab}
  mysql-server-core-5.5{ab} php5-cli{a} php5-common{a} php5-mysql{a}
  php5-snmp{a} php5-suhosin{a} rrdtool{a} x-ttcidfont-conf{a} x11-common{a}
  xfonts-encodings{a} xfonts-utils{a}

2) Setup your timezone in php.ini

There are three different "php.ini" files in Debian:

/etc/php5/cli/php.ini
/etc/php5/apache2/php.ini
/etc/php.ini

Cacti is using PHP over the CLI, so it should be enough if you update "/etc/php5/cli/php.ini", but I would update all of them. Add this line to the file:

date.timezone = "<time zone>"

For example:

date.timezone = "Asia/Tokyo"

The list of timezones is available here

3) Apache+MySQL+PHP+Cacti working out of the box with just a few commands

At this point you should be able to login in cacti on your apache webserver (http://VM_IP/cacti)

The default user/password is admin/admin. (the system will ask you to change it after the first login)

I won't explain Cacti configuration, because they already have very good documentation in their web site

4) Setting up the SSH authentication

This chapter is pretty well explained in Percona web site.

Basically we have to:
- Set up an SSH keypair for SSH authentication.
- Create a user on the remote server you want to graph.
- Install the public key into that user.s authorized_keys file.

Check the detailed information here Percona: SSH-Based Templates

5) Installing "Percona Monitoring Plugins for Cacti"

Download the plugins from Percona website: Percona - Download percona-monitoring-plugins

wget http://www.percona.com/redir/downloads/percona-monitoring-plugins/percona-monitoring-plugins-1.0.1.tar.gz

Extract the file, and copy ss_get_by_ssh.php to Cacti script folder:

tar zxvf percona-monitoring-plugins-1.0.1.tar.gz
cp percona-monitoring-plugins-1.0.1/cacti/scripts/ss_get_by_ssh.php /usr/share/cacti/site/scripts/

Edit /usr/share/cacti/site/scripts/ss_get_by_ssh.php with your remote user name and the path to the SSH identity from step 4.

$ssh_user   = 'cacti';                          # SSH username
$ssh_iden   = '-i /var/www/cacti/.ssh/id_rsa';  # SSH identity

6) Importing the template

Inside the tar file you will find different templates:

/percona-monitoring-plugins-1.0.1/cacti/templates# ls -lrt
total 1644
-rw-r--r-- 1 501 root 169078 Jun 15 04:44 cacti_host_template_percona_openvz_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  41926 Jun 15 04:44 cacti_host_template_percona_nginx_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root 843345 Jun 15 04:44 cacti_host_template_percona_mysql_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  89553 Jun 15 04:44 cacti_host_template_percona_mongodb_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  75491 Jun 15 04:44 cacti_host_template_percona_memcached_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  53416 Jun 15 04:44 cacti_host_template_percona_jmx_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root 249175 Jun 15 04:44 cacti_host_template_percona_gnu_linux_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  73879 Jun 15 04:44 cacti_host_template_percona_apache_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  37198 Jun 15 04:44 cacti_host_template_percona_redis_server_ht_0.8.6i-sver1.0.1.xml

In our case, we are only interested in cacti_host_template_percona_gnu_linux_server_ht_0.8.6i-sver1.0.1.xml

Console -> Import Templates -> Import template:

7) Create a new device

Under Console -> Devices, add a new device:

And complete the information (name, IP)

8) Modify the data template and the data source (I will use "Percona Disk Space DT" as example)

Check the Percona Data Templates availables:

Console -> Data Templates

Click in the name (Percona Disk Space DT) and let's edit the template. Go to the bottom of the page, and check "Custom Data":

Add Percona Disk Space DT as new data source for your host. Under Console -> Data Sources, click new:

Set the Remote IP and the partition (or volumen) to monitor

9) Create the graph

We have already created the data source for our remote server, so now, we just have to create a new graph.

Under Console -> Graph Management, add a new graph:

And select our data source (Percona Disk Space DT) for the device:

10) Result

At this point, the graph should be created, and after a few minutes, you should see something like this:

If something goes wrong, take a look at /var/log/cacti/cacti.log. There should be a line similar to this one:

<DATE> - CMDPHP: Poller[0] Host[3] DS[25] CMD: /usr/bin/php -q /usr/share/cacti/site/scripts/ss_get_by_ssh.php --host <REMOTE_IP> --type proc_stat --items gg,gh,gi,gj,gk,gl,gm,gn,go, output: gg:50994 gh:0 gi:76580 gj:56353167 gk:1617 gl:0 gm:0 gn:2135 go:-1

New house and new engine

psgonza — Sat, 16 Jun 2012 12:13:00 +0200

So, here we go again...

It's been a while since I moved this blog to blogger, and I think it's about time to move it back to a hosting I can manage.

New house

So I decided to move bynario.com to a VPS... There are very good deals outside, and you can get your own VPS for 10 euros per month. Obviously, the specs of the virtual machine you can get for that money won't allow you to run a site like Slashdot, but I don't expect too many visits.

I have to say that Blogger is a great platform. It allows you to use your own domain name, it's free, very reliable and really easy to use.

Actually, too easy. I am not very interested in blogging "per se" (don't have too much to say), so I see this more like a tool to stay tuned with new technologies, and who knows, someone could even find something interesting here.

New engine

Instead of using any of the popular blog engines available (usually based on Apache+PHP+MySQL), like Wordpress, Drupal, etc, I decided to give Octopress a try. You can find all the information about what it is and how to install it in Octopress website.

I still have to migrate all the post from Blogger (thanks to this post seems to be really easy) and customize the whole site a little bit. I don't know if this will take me hours, days, weeks or even years ;)

See you around...

Interesting talks (I)

psgonza — Sun, 03 Jun 2012 11:51:00 +0200

Sunday again...

Here you have a couple of interesting talks I found in Youtube today.

John Ousterhout - RAMCloud: Scalable High-Performance Storage Entirely in DRAM

https://www.youtube.com/watch?v=lcUvU3b5co8

Apache Hadoop - Petabytes and Terawatts

https://www.youtube.com/watch?v=SS27F-hYWfU

You can enjoy more videos in the LinkedIn Tech Talks channel in Youtube (LinkedInTechTalks)

New gadget: Super compact wireless router

psgonza — Sat, 02 Jun 2012 16:55:00 +0200

I am staying a few weeks in Japan, and it seems that here, most of the hotels still offer wired LAN as the only way to connect to the internet, instead of WiFi (I'm sorry, are you from the past?).

I usually travel with my smartphone and tablet (and a laptop of course), and they don.t have RJ45 connector, so I .need. a wireless network for them.

Last week, I spent a few hours walking around in Akihabara, the electronic district here in Tokyo, and I bought this cheap (around 24Eur) AP/wireless router: LAN-W300N/PR5 (by ELECON Logitec).

{% img center http://4.bp.blogspot.com/-35mvVxjrUwQ/T8CzVJDHRnI/AAAAAAAANzk/7Mqbvyc7w2U/s320/DSC_0167%5B1%5D 320 300 'Nice box. Pretty small' %}

Inside of the package you will find the power adapter (micro USB 5V/0.6A), a LAN cable (really short), and instructions (in a perfect Japanese):

{% img center http://2.bp.blogspot.com/-LI3SLjTILlw/T8CzGuGEIrI/AAAAAAAANzM/nN9r_v9lB7c/s320/DSC_0170%5B1%5D 320 300 %}

The device is really small, I would say it is the size of a old cassette tape. It has 4 LAN 10/100 ports, 1 LAN port for WAN and 2 USB ports (mini and micro USB) for the power supply:

{% img center http://3.bp.blogspot.com/-cwcArvxhERI/T8CzCorXsEI/AAAAAAAANzE/GExGOToIrGE/s1600/DSC_0171%5B1%5D 320 300 %}

The device is working just out of the box, acting as AP and as neutral router. This is the default configuration:

External network: DHCP client Internal network: LAN: 192.168.2.1/24 WLAN:
SSID: logitecuser Encryption: WPA2/PSK (You will find the password written under your device.)

Unfortunately, the router utility setup is completely in Japanese, which makes any change a little bit more difficult (but Google Translator does the job).

{% img center http://1.bp.blogspot.com/-GBxvqRC1NkA/T8oaCzDNi3I/AAAAAAAAN7s/LU1RJn1_BkY/s320/logitec_homepage.JPG %}

You can access the web setup using admin/admin

I have to say that this gadget is not meant to be a .top performer.. Actually, I had a few issues using Skype video-call, so keep that in mind before buying it. It is very cheap, and I would say that it is enough for web surfing and connecting your smartphone, tablet, microwave (-: when you are traveling, but not for being used on a regular basis.

Conclusions: PROS: Cheap, really portable, works with a USB (5V), 4 10/100 ports, and oh yes, cheap! CONS: Configuration utility in Japanese, average performance, not easy to buy outside Japan.

Sayonara!

Sometimes there is no need to complicated

psgonza — Thu, 24 May 2012 14:56:00 +0200

No images, no CSS, no fancy ajax or javascript files and more important no flash... Plain and simple text.

{% img center /img/lynx.JPG 320 300 'putty_cyg + cygwin 1.7.15 + lynx + wikipedia.org' %}

Software released in 1992 that keeps rocking (with limitations :P)

PS: I wouldn't recommend it if you want to check your G+ or FB profile

Like in the 90's

psgonza — Wed, 23 May 2012 14:35:00 +0200

This reminds me of the good old days, when upgrading Potato or Woody (Debian) took more than 5 minutes :)

{% img center /img/upgrading_cygwin.JPG 320 300 %}

Doesn't really matter which server I choose... Download speed sucks :)

Overwriting Windows keyboard shortcuts with AHK

psgonza — Tue, 17 Apr 2012 14:06:00 +0200

I was looking for an easy way to put my screen on sleep mode when I locked my laptop using WINDOWS - L, and also to mute the music player (in my case, foobar2000 or spotify)

By using AutoHotKeys, you can tweak your Windows keyboard shortcuts and make the most of them.

This is how I made it:

;###########################################################################
; LOCK WINDOWS & STOP FOOBAR || SPOTIFY & POWER OFF SCREEN (WIN - L) 
;###########################################################################

#l::
Run,%A_WinDir%\System32\rundll32.exe user32.dll`,LockWorkStation
Sleep 1500
SendMessage 0x112, 0xF170, 2,,Program Manager
Process, Exist, foobar2000.exe
If (ErrorLevel <> 0) ; If it is not running
{
Run "C:\Program Files\foobar2000\foobar2000.exe" /stop
}
Process, Exist, spotify.exe

If (ErrorLevel <> 0) ; If it is not running
{
DetectHiddenWindows, On
WinGetTitle, spotifyTitle, ahk_class SpotifyMainWindow
spotifyTitleLength := StrLen(spotifyTitle)
spotifyActive := InStr(currentTitle, "Spotify") 
spotifyPlaying := spotifyTitleLength > 7
if spotifyPlaying {
ControlSend, ahk_parent, {Space}, ahk_class SpotifyMainWindow
}
DetectHiddenWindows, Off
}

return

Working like a charm for me ;)

PERL: Checking Google Reader subscriptions status

psgonza — Tue, 21 Feb 2012 08:53:00 +0100

RSS subscriptions in my Google Reader account have reached an indecent number. I am subscribed to 398 sites (blogs, forums or whatever).

I will need to unsubscribe for a few of them I am no longer interested in, but I had the feeling that a lot of the feeds in my list weren't active at all.

Given the number of subscriptions, I wasn't too keen on checking each link manually, so I came up with this perl script.

I haven't use any module, so it could be executed with the default Perl environment that comes "out-of-the-box". The only requirements are wget, curl , the your subscriptions exported into a OPLM file and internet connectivity :)

I have tested it under Cygwin 1.7 and Arch Linux, but any other GN/Linux distro should work fine as well.

The execution is pretty straightforward:

$ perl check_google_subscriptions.pl google-reader-subscriptions.xml

The script will look for duplicated items in your file, and will go through all your subscriptions, looking for the http response.

This the source code:

#!/usr/bin/perl
#######################################################
#Author: psgonza
#Date: 19-Feb-2012
#Objetive: Check Google Reader feed links status
#Version:1
#######################################################
use FileHandle;
use strict;
use warnings;
binmode STDOUT, ":utf8";
$| = 1;
die "[ERR] Usage: $0 <file.xml>\n" unless @ARGV;
my $wget = system ("which wget > /dev/null 2>&1");
if ($wget ne 0) {print "[ERR] wget not found\n"; exit 0;}
my $curl = system ("which curl > /dev/null 2>&1");
if ($curl ne 0) {print "[ERR] curl not found\n"; exit 0;}
my $googlexml = $ARGV[0];
my @lines;
my @new_lines;


sub check_link{
        my $urlList = shift;
        my @wrongUrl = ();
        my $counter = 1;
        print "$counter";
        foreach my $url(@$urlList){
        my $result = system ("wget --spider \"$url\" --server-response --timeout=5 --tries=3 > /dev/null 2>&1");
        if ($result != 0) { push (@wrongUrl, $url); }
        if ($counter % 10) {print ".";} else {print "$counter";}
        $counter++;    
        }
        printf " DONE!\n\n";
        return \@wrongUrl;
}
print "==============================\n";
#Open xml file
open FILE, $googlexml or die $!;
#Create array with feeds urls
while (<FILE>)
{
        if ($_ =~ /xmlUrl=\"(.*)\" /) { push (@lines, $1);}
}
#Close xml file
close FILE;
my $total_items = scalar(@lines);
print "$total_items links found in $googlexml\n";
#Create hash with uniqe items
my %tmp = map { $_, 0 } @lines;
my @links = sort(keys %tmp);

#Check for duplicated items
print "\n===========================================\n";
print "Duplicated links:\n";
print "===========================================\n";
my %cnt;
$cnt{$_}++ for @lines;
print "$_\n" for grep $cnt{$_} > 1, keys %cnt;



my $items = scalar(@links);
print "\n==============================================================\n";
print "Checking server response for $items items. It can take a while...\n";
print "===============================================================\n\n";
my $ref_list = check_link(\@links);
my $deadlink = scalar(@$ref_list);
if ( $deadlink ne 0 )
{
        print "$deadlink problematic links found\n\n";
}
else
{
        print "\nAll links working\n";
        print "\n===========================================\n";
        print "DONE\n";
        print "===========================================\n";
        undef($ref_list);
        exit 0;
}
print "\n===========================================\n";
print "Those problematic links could fail due to:\n\n";
print "- RSS not available in the server any more\n";
print "- Spiders are blocked in Robots.txt so the query fails\n\n";
print "Downloading the headers using curl.";
foreach my $val(@$ref_list){
        print ".";
        chomp $val;
        if (my $fineresult = system ("curl --connect-timeout 5 --max-time 5 --dump-header headers.txt \"$val\" > /dev/null 2>&1 && (cat headers.txt | grep \"HTTP/1.1 200 OK\" > /dev/null)") != 0) 
                { push (@new_lines, $val); }
}
print ". DONE! \n\n";
$deadlink = scalar(@new_lines);
if ( $deadlink ne 0 )
        {
        print "\n===========================================\n";
        print "$deadlink links found not available\n\n";
        }
        else {
        print "All links working\n";
        }
foreach my $val(@new_lines){
        print "--> $val <-- NOT AVAILABLE\n";
}

undef($ref_list);
print "\n===========================================\n";
print "DONE\n";
print "===========================================\n";

And this how the output will look like:

$ perl check_google_subscriptions.pl example_google_export
==============================
60 links found in example_google_export
===========================================
Duplicated links:
===========================================
http://www.xtorquemadax.com/feeds/posts/default?alt=rss
==============================================================
Checking server response for 59 items. It can take a while...
===============================================================
1.........10.........20.........30.........40.........50......... DONE!
7 problematic links found

===========================================
Those problematic links could fail due to:
- RSS not available in the server any more
- Spiders are blocked in Robots.txt so the query fails
Downloading the headers using curl......... DONE!

===========================================
7 links found not available
--> http://blog.fredjean.net/articles.atom
--> http://blogs.elcorreo.com/el-navegante/posts.rss
--> http://feeds.feedburner.com/github
--> http://veejoe.net/blog/feed/
--> http://www.irfree.com/feed/rss/
--> http://www.misionurbana.com/site/index.php?q=rss.xml
--> http://www.rsdownload.net/option,com_rss/feed,RSS2.0/no_html,1.html
===========================================
DONE
===========================================

It's not big deal, but does the job!

Enjoy :)

TIP: Tweak your history command in Linux

psgonza — Sat, 18 Feb 2012 23:18:00 +0100

Note to self:

export HISTTIMEFORMAT='%F %T '
export HISTCONTROL=ignoredups
export HISTCONTROL=ignorespace
export HISTCONTROL=erasedups

Don't forget to add those lines to your ~/.bashrc...

Via Thegeekstuff

PHOTO: Cross Road in Whitehall (London)

psgonza — Fri, 17 Feb 2012 17:53:00 +0100

{% img center /img/DSC_0253.jpg 640 640 %}

Modelo: NIKON D5000
ISO: 200
Exposicion: 1/160 s
Apertura: 5.3
Longitud focal: 66mm
Flash usado: false

BLOGGER: Imposible importar xml

psgonza — Tue, 05 Jul 2011 10:15:00 +0200

Despues de estar peleando unos pocos de dias con los scripts del proyecto Google-blog-converters-appengine para convertir un export de entradas antiguas de Bynario, cuando usaba WordPress como CMS, ayer consegui que me funcionara....

He tenido que hacer 1000 perrerias a los ficheros XML, formatear datos, eliminar tags, cambiar juegos de caracteres, e incluso la mas dolorosa, borrar los comentarios (parece que no le gustaban los comentarios a los scripts). Pero al final los scripts funcionaron y pude crear un xml formateado para blooger.

Y cuando todo eso estaba funcionando, y llegaba la parte facil, subir el xml con el formato correcto, parece que no va a funcionar:

{% /img/importing_bynario.JPG %}

El paso 1, que es la verificacion del formato del fichero, lo ha pasado sin problemas, pero despues no pasa de ahi... Lleva unas horillas, y no se crea nada... Bien por mi y bien por Google

PHOTO: Toronto Skyline

psgonza — Sat, 02 Jul 2011 15:22:00 +0200

{% img center /img/DSC_0873.jpg 640 640 %}

Modelo: NIKON D5000
ISO:500
Exposicion: 2
Apertura: 4.5
Longitud focal: 26mm
Flash usado: false
Latitud: 43.623285
Longitud: 79.376537

30 years of FPS

psgonza — Thu, 24 Feb 2011 15:06:00 +0100

{% youtube PZqf5xqKgCE%}

PHOTO: London Sunset from Lambeth bridge

psgonza — Sat, 12 Feb 2011 18:21:00 +0100

{% img center /img/DSC_0268_69_70_tonemapped.jpg 640 640 %}

Modelo: NIKON D5000
ISO: 200
Exposició1/100 s
Apertura: 4.5
Longitud focal: 38mm
Lat/Long: 51.49486, -0.12325

NYC: Welcome to the '80s

psgonza — Mon, 31 Jan 2011 15:55:00 +0100

Aqui os dejo unas fotillos de NYC a principios de los 80's que he visto por gothamish.com...

Flashback New York City (1978-1980)

"TIP: Script para convertir Doc a RFT"

psgonza — Wed, 19 Jan 2011 15:41:00 +0100

He tenido que convertir todos los documentos doc que tenia en una carpeta a formato RTF (para que calibre me dejara pasarlos a distintos formatos).

Como eran unos pocos, en lugar de hacerlo a mano, he usado un script en VBS para ahorrarme un rato.

On Error Resume Next
Dim fso, folder, files, NewsFile,sFolder

Set fso = CreateObject("Scripting.FileSystemObject")
sFolder = Wscript.Arguments.Item(0)
If sFolder = "" Then
Wscript.Echo "No Folder parameter was passed"
Wscript.Quit
End If
Set NewFile = fso.CreateTextFile(sFolder&"\FileList.txt", True)
Set folder = fso.GetFolder(sFolder)
Set files = folder.Files

For each folderIdx In files
'NewFile.WriteLine(folderIdx.Name)
Doc2RTF folderIdx.Name 
Next
NewFile.Close


Sub Doc2RTF( myFile )
' This subroutine opens a Word document, then saves it as RTF, and closes Word.
' If the RTF file exists, it is overwritten.
' If Word was already active, the subroutine will leave the other document(s)
' alone and close only its "own" document.
'
' Written by Rob van der Woude
' http://www.robvanderwoude.com

' Standard housekeeping
Dim objDoc, objFile, objFSO, objWord, strFile, strRTF

Const wdFormatDocument = 0
Const wdFormatDocument97 = 0
Const wdFormatDocumentDefault = 16
Const wdFormatDOSText = 4
Const wdFormatDOSTextLineBreaks = 5
Const wdFormatEncodedText = 7
Const wdFormatFilteredHTML = 10
Const wdFormatFlatXML = 19
Const wdFormatFlatXMLMacroEnabled = 20
Const wdFormatFlatXMLTemplate = 21
Const wdFormatFlatXMLTemplateMacroEnabled = 22
Const wdFormatHTML = 8
Const wdFormatPDF = 17
Const wdFormatRTF = 6
Const wdFormatTemplate = 1
Const wdFormatTemplate97 = 1
Const wdFormatText = 2
Const wdFormatTextLineBreaks = 3
Const wdFormatUnicodeText = 7
Const wdFormatWebArchive = 9
Const wdFormatXML = 11
Const wdFormatXMLDocument = 12
Const wdFormatXMLDocumentMacroEnabled = 13
Const wdFormatXMLTemplate = 14
Const wdFormatXMLTemplateMacroEnabled = 15
Const wdFormatXPS = 18

' Create a File System object
Set objFSO = CreateObject( "Scripting.FileSystemObject" )

' Create a Word object
Set objWord = CreateObject( "Word.Application" )

With objWord
' True: make Word visible; False: invisible
.Visible = True

' Check if the Word document exists
If objFSO.FileExists( myFile ) Then
Set objFile = objFSO.GetFile( myFile )
strFile = objFile.Path
Else
WScript.Echo "FILE OPEN ERROR: The file does not exist" & vbCrLf
' Close Word
.Quit
Exit Sub
End If

' Build the fully qualified HTML file name
strRTF = objFSO.BuildPath( objFile.ParentFolder, _
objFSO.GetBaseName( objFile ) & ".rtf" )

' Open the Word document
.Documents.Open strFile

' Make the opened file the active document
Set objDoc = .ActiveDocument

' Save as HTML
objDoc.SaveAs strRTF, wdFormatRTF

' Close the active document
objDoc.Close

' Close Word
.Quit
End With
End Sub

Colocad el script en el directorio donde esten los ficheros.doc, y desde la consola de DOS ejecutad:

doc2rtf.vbs .

Saludos

"TIP: Script para creacion de usuarios en Linux"

psgonza — Fri, 14 Jan 2011 03:23:00 +0100

In case you need to define a lot of users in a Unix/Linux box, this small script could save you some time:

{% codeblock create_bulk_user.sh %} !/bin/bash file=.user_list.txt. for i in cat $file do user=echo $i | cut -d.;. -f1 pass=echo $i | cut -d.;. -f2 echo .Creating user: $user. useradd -m $user && echo .DONE. || echo .FAIL. echo .Creating entry in login.allow for user: $user. echo .$user all. >> /etc/login.allow echo .Setting the password for user: $user. echo .$pass. | passwd .stdin .$user. && echo .DONE. || echo .FAIL. echo .$user DONE. echo done

You just need to create the "user_list.txt" file containing your list of users/passwords in this format:

{% codeblock %} usuario1;pass1 usuario2;pass2

The script will create the user, set the password, and create an entry in allow.login

Bye!

TIPS: Instalar VMWARE en OpenSuse 11

psgonza — Sat, 01 Jan 2011 00:48:00 +0100

Para instalar alguna de las diferentes versiones de VMWARE (ya sea Player, Workstation, o Server) es probable que una vez instalada la aplicaciós pida unas ciertas dependencias para compilar los móos:

{% img center http://2.bp.blogspot.com/_2-x1cSWxK2M/TR5rKIgQq0I/AAAAAAAAL7M/AxDGVOaXijc/s1600/vmware_req.png 320 320 %}

En OpenSuse es tan facil ejecutar desde un terminal:

zypper in gcc kernel-source kernel-syms

Saludos