Blog engines comparation

Fri 17 October 2014

I don't like Ruby, at all... Maybe because I never really "used it" so didn't have the opportunity to know its guts, or maybe because it just sucks... If you ask me, it is probably the latter :P

As you might know, Octopress (which is a great blog engine, kudos to Brandon Mathis) is written in Ruby, so every now and then I have to deal with rvm, and all the ruby paraphernaliam when an upgrade breaks it in my VM.

Working in an environment you are not familiar with is not easy and every time it breaks, I have to spend some time trouble-shooting it, so I have decided to take a look at other available blog engines, compare them with Octopress, and migrate to a more "friendly" environment (hopefully)

So I compared these three alternatives: Octopress, Tinkerer and Hugo

Below you can see some facts and personal opinions about the three engines:

Octopress:
- Written in Ruby
- Based on Jekyll
- Ready to use out of the box. (version 3.0 in progress)
- Installation pretty well documented, but requieres setting up ruby environment
- Not that really fast... at all
- Very mature (it's been around for 3 o 4 years)
- Add-on plugins
- 8380 stars in GitHub

{% img center /img/octopress_Capture.JPG octopress %}

Tinkerer:
- Written in Python
- Based on Sphinx
- No markdown
- It needs a few dependences (like python-dev libxml2-dev libxslt-dev lib32z1-dev libz-dev)
- Fast
- 156 stars in GitHub

{% img center /img/tinker_Capture.JPG tinkerer %}

Hugo:
- Written Golang
- Using its own generation engine
- Highly customizable. Barely usable right after installation
- Run in multiple OS (including Windows)
- Really easy to install. Binary available for several OS
- Oh boy this is FAST.
- 2239 stars in GitHub

{% img center /img/hugo_Capture.JPG hugo %}

Besides installing and playing around with them, I decided to run a small benchmark, just to compare the performance of the different engines. Basically this is what I did:

1 - Generate 500 random posts:

$ time for i in `seq 1 500`; do rake new_post["post $RANDOM $RANDOM"];done
Creating new post: source/_posts/2014-10-15-post-25028-3011.markdown
mkdir -p source/_posts
Creating new post: source/_posts/2014-10-15-post-16477-16246.markdown
mkdir -p source/_posts
[...]
Creating new post: source/_posts/2014-10-15-post-1586-30872.markdown
mkdir -p source/_posts
Creating new post: source/_posts/2014-10-15-post-680-27529.markdown

real    7m35.910s
user    6m53.838s
sys     0m38.886s

$ time for i in `seq 1 500`; do tinker --post "post $RANDOM";done
New post created as '/root/bynario/2014/10/15/post_15124.rst'
New post created as '/root/bynario/2014/10/15/post_19047.rst'
[...]
New post created as '/root/bynario/2014/10/15/post_14905.rst'
New post created as '/root/bynario/2014/10/15/post_10370.rst'
New post created as '/root/bynario/2014/10/15/post_4049.rst'

real    2m18.046s
user    1m55.815s
sys     0m18.737s

$ time for i in `seq 1 500`; do hugo new post/post_${RANDOM}_${RANDOM}.md;done
/root/hugoblog/content/post/post_24649_29646.md created
/root/hugoblog/content/post/post_24308_8060.md created
/root/hugoblog/content/post/post_8311_6440.md created
[...]
/root/hugoblog/content/post/post_12973_1855.md created
/root/hugoblog/content/post/post_23049_13857.md created
/root/hugoblog/content/post/post_26136_21085.md created

real    0m15.123s
user    0m7.424s
sys     0m5.688s

2 - Add some text (no markdown, just 50 lines of plain text) to each post in order not to generate (almost) empty html pages

3 - Build the whole site

$ time rake generate
## Generating Site with Jekyll
identical source/stylesheets/screen.css
Configuration file: /root/octopress/_config.yml
            Source: source
       Destination: public
      Generating... done.
 Auto-regeneration: disabled. Use --watch to enable.

real    2m45.143s
user    2m41.834s
sys     0m3.172s

$ time tinker --build
Making output directory...
Running Sphinx v1.2.3
loading pickled environment... not yet created
building [html]: targets for 502 source files that are out of date
updating environment: 502 added, 0 changed, 0 removed
reading sources... [100%] master
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] master
writing additional files... search
copying static files... WARNING: favicon file 'tinkerer.ico' does not exist
done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded, 1 warning.

real    0m59.897s
user    0m57.792s
sys     0m1.328s

$ time hugo  --theme=liquorice --buildDrafts
498 of 498 drafts rendered
0 future content
498 pages created
0 categories created
0 tags created
in 2098 ms

real    0m2.136s
user    0m1.860s
sys     0m0.252s

Let's put the results in a table... I think figures speak for themselves:

	Octopress	Tinkerer	Hugo
Page generation	7m35.910s	2m18.046s	15.123s
Site generation	2m45.143s	59.87s	2.136s

(The result may vary running the test in bare metal instead of in a -crappy- VM, but still they are very descriptives)

Based on the results:

Ok, so Octopress is slow, and old... So what? It works really good out of the box, speed would be a problem if you post several times per day, but in my case, so it is not really an issue. For me the only thing that makes me think about changing the blog engine is Ruby... As simple as that
Tinkerer looks like a neat product, and I will problably take another look at it, but since it doesn't allow markdown, I am not so sure if I feel like migrating the post to a different format... Still, it is based in Python, and it's faster than Octopress, so I will give it a try...
Then should we move to Hugo blindfolded?? I don't think so... Not just yet. It is true that it is way faster, but I found it pretty complex for someone that doesn't really want to build the blog "almost" from scratch. Nevertheless, it looks really promising, it is surprisingly fast, and run in multiple OS (even from Windows)... So it might be a perfect alternative (at least for me) as soon as it is a little bit more mature.

Tricky decision...Should I stay or should I go?? :)

Bye!

Python: Parsing XML files

Tue 05 August 2014

{% img center /img/python-xml.jpg 'xml' %}

I would like to continue with another small python example. As in the previous post,this is more like a "note to self" thing than a educational post (this is not stackoverflow), but anyway, I guess it could be handy for someone(it has been for me...)

This time, I have been testing two xml submodules availables in Python 2.7 XML package:

xml.dom: the DOM API definition
xml.etree.ElementTree: the ElementTree API, a simple and lightweight XML processor

Both of them are pretty easy to use, and I haven't found any real difference in time execution (at least for basic filters) between both submodules.

I will use this xml file as an example:

<?xml version="1.0"?>
<data>
    <node>
        <attribute>Front End</attribute>
        <res_ids>100</res_ids>
        <nEName>BALDR</nEName>
        <ipList>192.168.0.5</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Web</attribute>
        <res_ids>12</res_ids>
        <nEName>THOR</nEName>
        <ipList>192.168.0.20</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Storage</attribute>
        <res_ids>200</res_ids>
        <nEName>VALI</nEName>
        <ipList>192.168.0.10</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Ubuntu</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>DB</attribute>
        <res_ids>230</res_ids>
        <nEName>LOKI</nEName>
        <ipList>192.168.0.110</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Fedora</so>
        <kernel>3.0.93</kernel>
    </node>
    <node>
        <attribute>Backup</attribute>
        <res_ids>300</res_ids>
        <nEName>OTHER</nEName>
        <ipList>192.168.0.103</ipList>
        <ipv6List>fe80::1:f6f1:fe01:12</ipv6List>
        <so>Debian</so>
        <kernel>3.0.93</kernel>
    </node>
</data>

In the tree, we can see that we have one "data" object containing several "nodes", each one with 7 "attributes".

Now let's extract the nodes and all the attributes using xml.dom:

#!/usr/bin/env python
from xml.dom import minidom

def parse_file(filename):
    xmldoc = minidom.parse(filename)
    for node in xmldoc.getElementsByTagName('node'):
        print str(node.getElementsByTagName("attribute")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("res_ids")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("nEName")[0].firstChild.nodeValue),
        print str(node.getElementsByTagName("ipList")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("ipv6List")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("so")[0].firstChild.nodeValue)
        print str(node.getElementsByTagName("kernel")[0].firstChild.nodeValue)

parse_file("./fakexml.xml")

And now, the same thing using xml.etree.ElementTree:

#!/usr/bin/env python
from xml.etree import ElementTree as ET

def parse_file(filename):
    tree = ET.ElementTree(file=filename)
    attrList = [ entry for entry in tree.findall('./node') ]
    for e in attrList:
        print e.find('attribute').text,
        print e.find('res_ids').text,
        print e.find('nEName').text,
        print e.find('ipList').text,
        print e.find('ipv6List').text,
        print e.find('so').text,
        print e.find('kernel').text

parse_file("./fakexml.xml")

The output from any of the two options would be the same one:

$> python print_table_from_xml_dom.py
Front End 100 BALDR 192.168.0.5 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Web 12 THOR 192.168.0.20 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Storage 200 VALI 192.168.0.10 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
DB 230 LOKI 192.168.0.110 fe80::1:f6f1:fe01:12 Fedora 3.0.93
Backup 300 OTHER 192.168.0.103 fe80::1:f6f1:fe01:12 Debian 3.0.93


$> python print_table_from_xml_element.py
Front End 100 BALDR 192.168.0.5 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Web 12 THOR 192.168.0.20 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
Storage 200 VALI 192.168.0.10 fe80::1:f6f1:fe01:12 Ubuntu 3.0.93
DB 230 LOKI 192.168.0.110 fe80::1:f6f1:fe01:12 Fedora 3.0.93
Backup 300 OTHER 192.168.0.103 fe80::1:f6f1:fe01:12 Debian 3.0.93

As I mentioned before, for this small exercise, there is no difference in terms of time of execution between both alternatives. I created a "slightly bigger" xml file (100x bigger), and the times were still pretty much the same, so I would say it is not a matter of the size, but the complexity of the filters.

To wrap up, let's put this code together with the one in the previous post, where we created a table from the data in a dictionary.

I use ElementTree instead of xml.dom, but both of them would work. Here's the code:

#!/usr/bin/env python
import sys
from collections import defaultdict
from xml.etree import ElementTree as ET

if sys.argv[1:]:
    inputF = sys.argv[1]
else:
    print "ERROR. Execution failed: missing file"
    sys.exit(1)

total = defaultdict(list)

# Column width
AttrColLen=25
ValueColLen=20
colwidth1="{0:<"+str(AttrColLen)+"}"
colwidth2="{0:<"+str(ValueColLen)+"}"


# Printer function
def print_table(data):
    for key, values in sorted(total.items()):
        print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"
        print  "| " + colwidth1.format(key) + "|",
        for i in xrange(len(values)):
            print colwidth2.format(values[i])+"|",
        print ""
    print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"

# Parse the xml, and add the items to the dictionary
def parse_file(filename):
    colwidth1="{0:<30}"
    colwidth2="{0:<25}"

    tree = ET.ElementTree(file=filename)
    attrList = [ entry for entry in tree.findall('./node') ]
    for e in attrList:
        total["Attribute"].append(e.find('attribute').text)
        total["RES_ID"].append(e.find('res_ids').text)
        total["networkElementName"].append(e.find('nEName').text)
        total["IPv4"].append(e.find('ipList').text)
        total["IPv6"].append(e.find('ipv6List').text)
        total["SO"].append(e.find('so').text)
        total["Kernel"].append(e.find('kernel').text)


parse_file(inputF)
print_table(total)

And here is the result:

$> python print_table_from_dict.py fakexml.xml
|----------------------------------------------------------------------------------------------------------------------------------------|
| Attribute                | Front End           | Web                 | Storage             | DB                  | Backup              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| IPv4                     | 192.168.0.5         | 192.168.0.20        | 192.168.0.10        | 192.168.0.110       | 192.168.0.103       |
|----------------------------------------------------------------------------------------------------------------------------------------|
| IPv6                     | fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12| fe80::1:f6f1:fe01:12|
|----------------------------------------------------------------------------------------------------------------------------------------|
| Kernel                   | 3.0.93              | 3.0.93              | 3.0.93              | 3.0.93              | 3.0.93              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| RES_ID                   | 100                 | 12                  | 200                 | 230                 | 300                 |
|----------------------------------------------------------------------------------------------------------------------------------------|
| SO                       | Ubuntu              | Ubuntu              | Ubuntu              | Fedora              | Debian              |
|----------------------------------------------------------------------------------------------------------------------------------------|
| networkElementName       | BALDR               | THOR                | VALI                | LOKI                | OTHER               |
|----------------------------------------------------------------------------------------------------------------------------------------|

;)

Python: print table from dictionary of lists

Thu 31 July 2014

{% img center /img/python-logo-inkscape.jpg 'python' %}

Hello

I have been trying to find an easy way to create ascii tables in Python using data from a dictonary of lists. I couldn't find anything that suited me, so I came up with this small piece of code

{% codeblock %} def print_table(data): for key, values in sorted(total.items()): print "|" + AttrColLen"-" + ((ValueColLen+2)len(values))"-" + "-|" print "| " + colwidth1.format(key) + "|", for i in xrange(len(values)): print colwidth2.format(values[i])+"|", print "" print "|" + AttrColLen"-" + ((ValueColLen+2)len(values))"-" + "-|"

It's pretty simple, and does the job. Let me show it to you with an example

#!/usr/bin/env python
from collections import defaultdict

total = defaultdict(list)

# Column width
AttrColLen=25
ValueColLen=15
colwidth1="{0:<"+str(AttrColLen)+"}"
colwidth2="{0:<"+str(ValueColLen)+"}"

# Fake data 
attribute=['Front End','Web','Storage']
res_ids=['100','12','200']
nEName=['BALDR','THOR','VALI']
ipList=['192.168.0.5','192.168.0.20','192.168.0.10']

# Printer
def print_table(data):
    for key, values in sorted(total.items()):
        print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"
        print  "| " + colwidth1.format(key) + "|",
        for i in xrange(len(values)):
            print colwidth2.format(values[i])+"|",
        print ""
    print "|" + AttrColLen*"-" + ((ValueColLen+2)*len(values))*"-" + "-|"



# Populating the dictionary
for iter1 in attribute:
    total["Attribute"].append(iter1)
for iter2 in res_ids:
    total["RES_ID"].append(iter2)
for iter3 in nEName:
    total["networkElementName"].append(iter3)
for iter4 in ipList:
    total["IPv4"].append(iter4) 

# Let's print the dictionary
print total

Obviously there are other (better!!) ways to print a dictionary (like pprint),anyway I just wanted to show you how the dictionary looks like

defaultdict(<type 'list'>, {'Attribute': ['Front End', 'Web', 'Storage'],
'RES_ID': ['100', '12', '200'], 
'networkElementName': ['BALDR', 'THOR', 'VALI'], 
'IPv4': ['192.168.0.5', '192.168.0.20', '192.168.0.10']})

Now let's replace:

# Let's print the dictionary
print total

By:

# Let's print the dictionary
print_table(total)

This is the result:

|-----------------------------------------------------------------------------|
| Attribute                | Front End      | Web            | Storage        |
|-----------------------------------------------------------------------------|
| IPv4                     | 192.168.0.5    | 192.168.0.20   | 192.168.0.10   |
|-----------------------------------------------------------------------------|
| RES_ID                   | 100            | 12             | 200            |
|-----------------------------------------------------------------------------|
| networkElementName       | BALDR          | THOR           | VALI           |
|-----------------------------------------------------------------------------|

It looks pretty nice, and the code is really simple...

I hope it helps

Byte!

Network manager for windows 7 using cmd.exe

Sun 16 February 2014

{% img center /img/cmd_exe.JPG 'cmd.exe' %}

Updated version of the script. Now it allows you to enable/disable the network interfaces

Hi there

Lately I have been changing the network properties (ip addresses, netmask, etc etc) in my laptop (W7 Enterprise 64b) several times per day, and to be honest I am growing tired of it, so I created a small bat script in order to make my life easier

The script is pretty simple... It might be handy for you as well, so here you are:

@ECHO OFF
SET OPTION=

goto :MENU

:MENU
cls
Echo #### CLI NETWORK MANAGEMENT ####
Echo.  
Echo ****** LAN ******
Echo 1) LAN - DHCP
Echo 2) LAN - STATIC IP:192.168.2.10/24 - GW:192.168.2.99 - DNS:8.8.8.8
Echo 3) LAN - DEFINE NEW STATIC IP
Echo.
Echo ****** WIRELESS LAN ******
Echo 4) WLAN - DHCP
Echo 5) WLAN - STATIC IP:192.168.2.20/24 - GW:192.168.2.99 - DNS:8.8.8.8
Echo 6) WLAN - DEFINE NEW STATIC IP
Echo.
Echo ****** INTERFACES ******
Echo 7) SHOW INTERFACES STATUS
Echo 8) ENABLE LAN
Echo 9) DISABLE LAN
Echo 10) ENABLE WLAN
Echo 11) DISABLE WLAN
Echo.
Echo 0) Exit
Echo. 
set /p OPTION=Option: 
Echo. 

IF "%OPTION%"=="0" call :END
IF "%OPTION%"=="1" call :Opt1 "Local Area Connection"
IF "%OPTION%"=="2" call :Opt2 "Local Area Connection" "192.168.2.10" "255.255.255.0" "192.168.2.99" "8.8.8.8"
IF "%OPTION%"=="3" call :Opt3 "Local Area Connection"
IF "%OPTION%"=="4" call :Opt1 "Wireless Network Connection"
IF "%OPTION%"=="5" call :Opt2 "Wireless Network Connection" "192.168.2.20" "255.255.255.0" "192.168.2.99" "8.8.8.8"
IF "%OPTION%"=="6" call :Opt3 "Wireless Network Connection"
IF "%OPTION%"=="7" call :Opt4 
IF "%OPTION%"=="8" call :Opt41 "Local Area Connection" "ENABLED"
IF "%OPTION%"=="9" call :Opt41 "Local Area Connection" "DISABLED"
IF "%OPTION%"=="10" call :Opt41 "Wireless Network Connection" "ENABLED"
IF "%OPTION%"=="11" call :Opt41 "Wireless Network Connection" "DISABLED"


goto :MENU

:Opt1
REM DHCP
REM "%~1" - Interface Name
cls
Echo Configuring %~1 as DHCP...
Echo. 
Echo Configuring DHCP for IP
netsh interface ip set address "%~1" dhcp
Echo Configuring DHCP for dns
netsh interface ip set dns "%~1" dhcp
Echo Configuring DHCP for wins
netsh int ip set wins name="%~1" dhcp
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
Echo. 
pause
goto :MENU


:Opt2
REM STATIC IP
REM "%~1" - Interface Name
REM "%~2" - IP
REM "%~3" - NETMASK
REM "%~4" - GW
REM "%~5" - DNS
cls
Echo Configuring static IP address for %~1
Echo =============================================
Echo.
Echo Configuring IP %~2 %~3 %~4
netsh interface ip set address name="%~1" static "%~2" "%~3" "%~4"
Echo Configuring DNS %~5
netsh interface ip set dns name="%~1" static "%~5"
netsh interface ip add dns name="%~1" 8.8.4.4
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
pause
goto :MENU

:Opt3
REM STATIC IP - NEW
REM "%~1" - Interface Name
cls
set /p IP=IP:
set /p NET=NETMASK:
set /p GW=GATEWAY:
set /p DNS=DNS:
Echo Configuring new static IP address for %~1
Echo =============================================
Echo.
Echo Configuring IP %IP% %NET% %GW%
netsh interface ip set address name="%~1" static "%IP%" "%NET%" "%GW%"
Echo Configuring DNS %DNS%
netsh interface ip set dns name="%~1" static "%DNS%" 
Echo Configuring secondary DNS 8.8.4.4
netsh interface ip add dns name="%~1" 8.8.4.4
Echo. 
Echo Checking configuration
netsh interface ip show config name="%~1"
pause
goto :MENU

:Opt4
REM INTERFACES
REM "%~1" - Interface Name
REM "%~2" - status
cls
Echo ****** INTERFACES ******
Echo. 
Echo Checking network interface status...
Echo.
netsh interface  show interface | find /i "connection"
Echo.
pause
goto :MENU

:Opt41
cls
Echo ****** INTERFACES ******
Echo. 
Echo Setting %~1 status to ... %~2 
netsh interface set interface name="%~1" admin="%~2" 
goto :Opt4


:END
exit

The script needs to be executed as administrator...

Online again

Mon 27 January 2014

It's been a while.... but better late than never, isn't it? I am back online (I never left, but this site didn't come back after a VPS migration)...

Not a single post in 2013... "Working, working and working" that would sum up the whole last year.

These are my thoughts about a few things that happened in 2013:

RaspberryPi:

I got mine (pre-ordered) in the fall of 2011... Well actually I paid it then, but it was delivered in Jun 2012. In 2013 I bought another one to use it as a media center with raspbmc. What a piece of HW for only 37 euros.

Python:

I know I know, Python is OLD, but believe or not, I never used it before until 2013. I am not a developer, but Bash and Perl gave me pretty much everything I needed... Until I started to play with Python, it's easy, fast, and there are modules for almost everything you can think of.

Ansible:

According to them, "a radically simple IT orchestration solution", and you bet it is. Give it a chance if you can. Way easier than Puppet/Chef/Fabric...

I think the number of users in Google+ is ramping up mainly because it's mandatory if you want to use some Google products. Unfortunatelly, it didn't reach my expecations. There are some interesing stuff, but I am kind of bored of seeing animated gifs.

New family member:

After a few years using "old" computers (and my company laptop), I decided to buy a Lenovo x230, together with a 256GB Samsung SSD 840 PRO. To put is simple, it is F.A.S.T. As usually, GNU/Debian is the OS of choice. Still, some days I have second thoughs about it. Should I have bought a MBA 11" or this one?? I haven't reach any conclusion yet :)

Bye bye Google Reader

At the end, it wasn't a big deal. I had been a G.Reader power user for years, but due to my limited time, I started to check my feeds in the tablet (Nexus 7 2012) instead than in the laptop, and moving to Feedly was a smooth migration. You know what they say... the king is dead, long live the king.

Iceland and again to the US:

I did spent the whole year working long hours and weekends, but at least, I could take a few days off (in two different periods) and visited Iceland and some cities in the US East Coast. Iceland is amazing, it is like Jurasic Park (geologically speaking). You should totally go.

{% img center /img//DSC_0597.jpg 'Iceland' %}

And again, back to the US... We had a really good time, as usual.

{% img center /img//DSC_0600.NEF.jpg 'NYC' %}

{% img center /img//DSC_1057.NEF.jpg 'chicago' %}

I will try to write a few post this year, but I can't promise anything

//Bye

← Older Newer →