Watching Mr Robot Episode 1 (I know, I am kind of late, but I have a lot of TV series in my "backlog") I noticed again they used a wrong IPv4 address... Yes, they talk about rootkits, KDE, DDos, etc, etc, but one of the key details of this first episode it's this IP address (not the address itself, but what Malek does with it, if you know what I mean...). How come it's so difficult to remember that only values from 0 and 255 are allowed??? (we could discuss whether they are valid IP addresses for hosts or networks but let's keep it simple)

'218.108.149.373'

And unfortunately this simple error is pretty common in TV shows and movies... The same thing happened several months ago in Blackhat, another "hacking" movie:

'157.257.273.12'

I cannot wait to see what's going to happen when IPv6 finally kicks in :)

(I wrote this post in my way back to the old continent... According to the map, we were flying over Jarenga river, in the mother Russia)

I don't know you, but I have been using RSS clients since, well, foverer I'd say... First Akregator in KDE, then Google Reader (I still wonder why Google had to shut it down) and finally, as most of the people did, I moved to Feedly, and I am pretty happy with it, no complains at all.

A few days ago I was taking a look at my RSS subscriptions and there were more than 400 websites that I have been adding during all these years, so I deciced to clean it up a little bit using Python.

I came up with the script you can see below... It is fairly simple (and it uses only standard library modules) but does the job for me.

In order to use it, you just need to export your feedly subscriptios (you will get an opml file) and execute the script. You will get a new file without the "dead" links ready to be imported.

Usage:

# python3 feedly_checker.py myfeedlyrss.opml

Code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

#!/usr/bin/python3
import os
import sys
from xml.dom import minidom

import http.client
from urllib.parse import urlparse

errors = []

def usage():
    print("Usage: python3 " + sys.argv[0] + " <feedly_export_file>.opml")
    sys.exit(0)


def parse_feed(xmlfile):
    feeds = []
    for _ in range(len(xmlfile)):
        tmp = {}
        if xmlfile[_].getAttribute('type') == "rss":
            tmp[xmlfile[_].getAttribute('text')] = xmlfile[_].getAttribute('xmlUrl')
            feeds.append(tmp)
    return feeds


def check_http_response(url):
    for name, rss in url.items():
        addr = urlparse(rss)
        try:
            conn = http.client.HTTPConnection(addr[1], 80, timeout=3)
            conn.request("HEAD", addr[2])
            res = conn.getresponse()
            if res.status >= 400:
                errors.append(rss)
                print("Name: %s, URL:%s, Error Code:%d" % (name, rss, res.status))
        except http.client.HTTPException:
            errors.append(rss)
        finally:
            conn.close()


def load_xml(filename):
    long_input_file = os.path.realpath(filename)
    try:
        xmldoc = minidom.parse(long_input_file)
        item_res = xmldoc.getElementsByTagName('outline')
    except Exception:
        print("Error parsing xml file")
        sys.exit(1)
    return item_res


def clean_xml(filename, errors):
    xmldoc = minidom.parse(filename)
    doc_root = xmldoc.getElementsByTagName('outline')

    for j in doc_root:
        if j.getAttribute('type') == "rss" and j.getAttribute('xmlUrl') in errors:
            j.parentNode.removeChild(j)
            print("Removing... %s" % (j.getAttribute('xmlUrl')))

    try:
        f = open(filename + "_clean", "w")
        f.write(xmldoc.toxml())
        print("Creating %s_clean file..." % filename)
    except IOError:
        print("Error creating %s_clean file" % filename)
    else:
        f.close()

if len(sys.argv) != 2:
    usage()
else:
    input_file = sys.argv[1]

outlineItems = load_xml(input_file)

results = parse_feed(outlineItems)

print("%d entries found in %s" % (len(results), input_file))

for site in results:
    check_http_response(site)

print("%d wrong entries found" % (len(errors)))

clean_xml(input_file, errors)

sys.exit(0)

It's been a quick excersice (POC?), so there is a lot of room for improvement...

1. It is not fast, because it is single-thread, and it has to wait for the http connection to timeout.
2. Any error code above 400 will make the link to be removed... Some error codes 5xx are "temporary server errors", but they will be deteled as well... It'd need some extra work

\\psgonza

Disclaimer: it's summer, we are under a really long heat wave in Spain, so I didn't feel like moving from the couch (you will understand later :D)

Yesterday evening I was laying on the couch surfing Amazon website using an old laptop (very old) that I barely use, and at some point, I took an screenshot I needed to share with a friend via corporate email... (my idea was to send it to my email account, and this morning, send it from my corporate account)

Anyway, after taking the screenshot I realized:

• I was no logged in gmail, dropbox, github, etc , and in order to login, I would need a token from my cellphone (and my smartphone was in a different room!!!)
• No USB drives around
• The image sharing alternatives that I know of, requires you to create an user and login… Too much for a onetime thing

So I decided to use pastebin to store the image in text format:

1. Take the screenshot (I used scrot, feel free to use any other tool…) : scrot -d 5 ~/amazon.jpg

2. Gzip the file: gzip ~/amazon.jpg

3. Convert to base64: base64 ~/amazon.jpg.gz > ~/amazon.b64

4. Cat the file and paste the (long) result into pastebin and write down the url

This morning from my corporate laptop:

1. Download the file from pastebin (save it as amazon.b64)

2. Set file format UNIX... For instance, using vi: ESC:set ff=unix

3. Decode it:base64 -d amazon.b64 > amazon.jpg.gz

4. Unzip it: gunzip amazon.jpg.gz

Et voila, we have the same picture….

I know this doesn't seem very useful (or yes, who knows?) but you might follow the same approach for other things… For instance, I use pretty much the same thing in Python for embedding an old version of pexpect module in my scripts

Later

Hi there

After a few months in the shelf, today I reconnected my Synology DS215j, and to my surprise, I wasn't able to login via DiskStation GUI due to a problem with my autenthication:

{% img center /img/synology_ntp_mismatch.jpg 'DSM Error' %}

It was weird because:

a) (you are going to love this one...) "It was working before"
b) I could login to my NAS via SSH with the same user/pass, so it was clearly a problem with the 2 Factor Authethication

Once in SYNOLOGY CLI, I realized the system time was wrong:

{% img center /img/synology_2fa_error.png 'ntp mismatch' %}

As you can (hopefully) see in the picture, the time in the SYNOLOGY CLI (23:58:13) differs from the one in web GUI (23:54) (and the system time in my MBP as well)

So I connected to the NAS with root password (rather sooner than later I would need to "enable" sudo and block root user in SSH) and updated the date by running:

ntpdate -u pool.ntp.org

(If this isn't working, try killing ntpd and run ntpdate -u pool.ntp.org again)

After that, I managed to login using my 2FA code as expected...

Bytes!

PS: First post using github as text editor... Let's see how this goes.

While trying to "pip install ipython[notebook]" in my Cygwin installation (W7), I got this error due to libzmq:

 Collecting ipython[notebook]

[...]

 Running setup.py install for pyzmq

[...]

gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -ggdb -O2 -pipe -Wimplicit-function-declaration 
-fdebug-prefix-map=/usr/src/ports/python3/python3-3.4.3-1.i686/build=/usr/src/debug/python3-3.4.3-1 
-fdebug-prefix-map=/usr/src/ports/python3/python3-3.4.3-1.i686/src/Python-3.4.3=/usr/src/debug/python3-3.4.3-1 
-DZMQ_USE_POLL=1 -DHAVE_LIBSODIUM=1 -Ibundled/zeromq/include -Ibundled -Ibundled/libsodium/src/libsodium/include 
-Ibundled/libsodium/src/libsodium/include/sodium -I/usr/include/python3.4m -c bundled/zeromq/src/address.cpp 
-o build/temp.cygwin-2.0.4-i686-3.4/bundled/zeromq/src/address.o

gcc: error: spawn: No such file or directory
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-l4nkh9zq/pyzmq/setup.py';
exec(compile(getattr(tokenize,'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" 
install --record /tmp/pip-jicpjqak-record/install-record.txt --single-version-externally-managed --compile" 
failed with error code 1 in /tmp/pip-build-l4nkh9zq/pyzmq

It took me a while to figure it out... In case you were in a similar situation, this is what I did:

1) Install some tools (via apt-cyg or setup.exe)

apt-cyg install libtool automake autoconf   

(you will need the usual tools for compiling C/C++ code, such as gcc,g++,make, etc,etc)

2) Export PKG_CONFIG_PATH

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

3) Download and compile libsodium

./configure
make && make install

4) Download and compile libzmq

./autogen.sh
./configure
make && make install

5) Finally, try to install ipython[notebook] again:

pip install ipython[notebook]

It worked for me... Hope it helps you.

Later