~stef/blog/

Herr Nilsson is a bot which fetches data from parltrack and imports it into a mediawiki. This helps to improve the stubs on euwiki itself.

This also allows other organizations to run their own internal wiki, which can contain private analysis and commentary. A much requested feature. All you need is a mediawiki and Herr Nilsson is setting up stubs for the dossiers of interest.

On parltrack there's now a Preferences menu in the top blue bar, where you can set the address to your mediawiki, and parltrack will automatically display a Notes link in the top blue bar, which links to the dossiers page on your own hosted wiki.

Good news everyone, i just enabled hu, da, de, es, fi, fr, it, nl, pt, ru, se language support in pippi.

Clicktracking is evil, dnet and endre specified the details of an anonymous URL unshortening service (UUS), anonshort. Basically it resolves HTTP and HTML meta redirects, and cleans out those annoying Urchin Tracker Module URL parameters.

We currently don't provide a web user interface, only a very slim web API. Simply construct an URL by appending the shortened URL to http://anonshort.hsbp.org:8080/?u= and get a resolved URL back. Easiest is in the command line with curl:

curl 'http://anonshort.hsbp.org:8080/?u=<URL>'

the same service is also available for even more privacy as a tor hidden service at:

http://ixzr427vwpmxk3io.onion

using this tor hidden service is similarly easy with curl and torify:

torify curl 'http://ixzr427vwpmxk3io.onion/?u=<URL>'

We do cache the results, but we do so in a way that prohibits even us to deduce the input and output URLs without knowing the input URL. The algo is quite nifty i hope it stands up to scrutiny (check out cache.py).

All resolves are running over tor, the user agents are chosen from a pre-selected set of samples taken in the wild.

There was talk of a web-interface as well, let's see how that evolves.

enjoy it!

I experiment with omnom to use it also as a platform to announce updates to itself, check out the omnom-announcement tag or point your rss reader at the atom feed.

The latest message in a nutshell: for anyone who uses the userscript for bookmarking should update.

Good news everyone! Omnom - my feeble attempt at creating a proper^Wlibre delicious replacement - now has gotten "widget" functionality. I took the original delicious widget and shamelessly adopted it. You can see the result in the right bar under "/dev/read".

If you are one of the lucky omnom users, you can use the code below, just change the 2 links pointing to my collection to your own.

<h3><a href='http://links.ctrlc.hu/u/stf'>/dev/read</a></h3><div id="omnom-box" style="margin:0;padding:0;border:none;"> </div>
<script type="text/javascript" src="http://links.ctrlc.hu/u/stf/?format=json&j"></script>
<script type="text/javascript">
   var ul = document.createElement('ul');
   for (var i=0, post; post = omnom_posts[i]; i++) {
      var li = document.createElement('li');
      var a = document.createElement('a');
      a.setAttribute('href', post.url);
      a.appendChild(document.createTextNode(post.title));
      li.appendChild(a);
      ul.appendChild(li);
   }
   ul.setAttribute('id', 'omnom-list');
   document.getElementById('omnom-box').appendChild(ul);
</script>

Parltrack is a free tool and the associated free database to track the law-making in the European Parliament. It

• aggregates and publishes dispersed information,
• provides email and RSS based notifications,
• links most documents with Pippi Longstrings for analysis and commenting.

Combined information

Parltrack collects the data daily and combines the information on MEPs, progress on dossiers from OEIL, committee agendas, and vote results from the plenary minutes. Using this data it is easy to see for example at a glance:

• which MEP has taken responsibility for which dossiers,
• which committee is responsible or gives opinion on which dossiers,
• forecasts for next steps on dossiers in the EP and committees - including tabling deadlines for amendments,
• online detailed committee agendas,
• online calendar with EP and committee dates,
• who are the influential MEPs related to a dossier. There's much more possibilities in this data than the web-interface and your humble developers can ever present to you. Though we are willing to work for hire to dig deeper into this information-goldmine. ;)

Notifications

Anyone can subscribe to receive email notifications on changes on dossiers. On every dossier page, there's a link "Track this Dossier", by providing your email here you create a new notification group. If you don't specify a name of the notification group, one will be randomly generated for you. If you specify an existing notification group name, the current dossier will be added to that. After you created a notification group, you can share the link to this so other people can join in.

The Texts

Most of the documents created in the law-making process (initial, supplementary and final texts, opinions) are linked whenever available from OEIL. Most of these documents are also available on Eur-Lex (the official EU website publishing the texts) and thus are also automatically available in Pippi Longstrings from the Documents tab for dossiers. Using Pippi Longstrings you can analyse the texts and comment on them. Commenting happens by selecting the text you want to comment on. The results of pippi-analysis are also displayed as comments, from a user named Pippi Longstrings.

The Data

All data in Parltrack is available freely in various forms for anyone to use. Most of the pages have a "Download as JSON" link, all new and changed dossiers are listed in an RSS feed, changes in dossiers are also available as email notifications. Furthermore a complete DB dump as JSON is also available. All this is under the ODBLv1.0 license. Running your own instance of Parltrack is encouraged, the code is free under the AGPLv3+.

Searching

Currently search is possible for text in MEPs names, dossiers titles and IDs of final acts like "directive 2006/24/EC".

Credit

Parltrack is inspired by ehjs Tratten, was conceived during the 2010 FCForum Tools for policy reformers workshop and is influenced by a bunch of good people, thanks!

If you'd like to support similar tools in the future, don't hesitate to show your support:

Thanks, It's much appreciated.

i figured code is a mightier sword than mere words. please forgive me for focusing on forging instead of mumbling:

• burnstation2: a tool to educate people that file-sharing can be legal, a fork of pyjama. We needed some software to access Jamendo, pyjama was perfect, together with a touchscreen we have a nice burnstation. (kudos to the original pyjama author(s)!)
• playful swarm simulation - a game for evolutionary pythonistas. Also a product of h.a.c.k. several of us modeled and visualized simple swarm behavior, it's a lot of fun to tweak the config to get the most stable setup. (thx, dnet+asciimooo)
• ksh-scraper: liberating the data of the Hungarian Statistical Office. Our contribution to the Open Government Data Hackday December 2010, using a simple greasemonkey script (install) all statistical data on the Hungarian Statistical Office are also available as CSV downloads (shouts to asciimoo).
• memopol2: i'm contributing to this important project, which gets especially interesting when combined with weurstchen and pippi. (greetz lqdn!)
• django-mongo-annotator: annotator rocks, but didn't support django/mongo out of the box. not anymore!

And some other important projects, which i will address in separate posts:

• omnom (formerly known as tagger, tagr): capitalizing on the faked demise of del.icio.us, an attempt to get it right and free, including snapshoting of bookmarked pages. (live demo) (thx smari)
• weurstchen: the EU legislative machinery is always churning out new law, weurstchen monitors this. (live demo) (thx smari)
• deaddrop: responsible leaking for reckless copycats. anonymous delivery services for everyone else.
• pippi longstrings is alive, automatically importing eur-lex, etherpad and co-ment documents, and allows commenting using annotator. (live demo)

if you like this, please consider flattring me.

I hope these projects make up for not posting here. If not, here: have some shiny distracting movies generated by the swarm simulation: (HD:mp4,SD:ogv) of swarms (greens=algae, white=fish, red=shark):

Download Video: Closed Format: "MP4" Open Format: "Ogg"

Download Video: Closed Format: "MP4" Open Format: "Ogg"

Download Video: Closed Format: "MP4" Open Format: "Ogg"

Download Video: Closed Format: "MP4" Open Format: "Ogg"

Here's a new project, some of you might have already heard about it: Pippi Longstrings. Just like Bonobo (currently also off-line) this is part of le(n)x a set of tools to empower citizens in the legislative process. We're awaiting replacement of unstable hardware, but hope to have the problems sorted out soon and can start operations on http://pippi.euwiki.org. Until then there are some cached results from various stages of the development (the newest ACTA is a good example), so please excuse the varying quality of the docs in that cache.

Background

The original idea came from our team-member Erik Josefsson. Unfortunately he is not a lawyer, so he came up with the idea to make legislative texts (laws) more comprehensible for non-politicians, by looking for text-blocks that are copied from one document to another. These text-fragments act as memes that carry the most important legislation into new laws and other legal documents. By classifying these fragments and 'translating' them into short summaries, it is possible to ease the burden of reading such documents. Reverse-engineering the EU is something akin to having only the legal code of a Creative Commons license, but not the deed (a simple one line explanation - see an example for a deed) nor the icons, using Pippi we try to reduce the code into deeds and possibly also icons for easier understanding for citizens, activists, advisors and politicians themselves.

Just to give you an example of the above, the deed of the previous paragraph could say: "we look for copy/pasted texts (the longer the more interesting - hence the name: Pippi Longstrings) and try to translate this into short, comprehensible summaries".

Tracing sources

Among other this can also be used to track the sources of such fragments. Previously there have been interesting pippies discovered by manual inspection:

• the FSFE found out, that parts of the European Interoperability Framework have been "authored" by the Business Software Alliance.
• parts of the EU Telecom Package have been written by Telecom Italia (Amendment 542)
• and parts of EU laws (like the IPRED directive) have been included in Trade Agreements with Canada, Korea, scores of Caribian countries and possibly also India (more on this later).

There is a benefit for policy advocates as the above examples show. For non-legislative purposes Pippi Longstrings can also be used for tracking and translating memes in contracts, terms of service agreements and EULAs, similarity to the EFF's great Tosback service.

National implementations

Another important use-case for Pippi Lonstrings in regard to EU laws is the analysis of adoption of these laws into member states law. All EU laws are automatically translated and published in all 23 languages of the member states. So when a member state adopts a law, we can check whether they adopted the verbatim translation or changed bits and pieces while adopting it into national law. In the later case it is definitely interesting to analyze the reasons for the deviation from the original EU translation.

Current status

Pippi Longstrings is currently running as a closed beta. If you're interested in analyzing a certain document against the current European corpus of regulations and directives please suggest mail them to longstrings on ctrlc.hu.

We have about 40 documents that are on our list waiting to be processed. These are related to Internet, privacy, copyright topics and trade agreements, but we are looking for more docs to analyze.

Currently the processing of a doc takes a couple of hours against the whole EU corpus of law, the list of docs to be processed is being prioritized by us, until we succeed in adding a feature for user initiated processing and/or get donation for lots of powerful hardware. As an alternative, you can get the code which is completely free according to the Affero GNU Public License and operate a Pippi service yourself.

Document formats

Pippi Longstrings is part of a set of tools to reduce entry-barriers to participation in the European legislative process. Even though the European legislative process is obliged to be transparent, some - some of the most important - issues are shrouded in secrecy and so we need to rely on low quality PDF leaks for analyzing and reacting to them. These leaks are usually scanned PDFs which do not lend themselves for automated analysis. There is a grave need for solutions that are able to transform these PDFs into high-quality machine-processable documents (semantically correct HTML or ODF preferably). Currently we rely on crowd-sourced transcriptions mostly done by the Telecomix crew and La Quadrature du Net, but the EU-India trade agreement leak has not been transcribed yet by anyone - while it is surely a very interesting document. If anyone can help us to get these transcriptions more effectively, please share your tools, resources or whatever. It would be nice to have transcriptions done by recaptcha for example, google are you reading this?

If we have such a transcription or the original document is not a PDF, then we are able to produce some nifty diffs between different versions of these texts (think ACTA) so that we can track the negotiations without being admitted to them. The generation of these diffs is not as easy as you might know from software development though. Producing such diffs has involved a lot of manual labor to align the paragraphs properly for the most comprehensible results. The preprocessing prior to a diff for a typical ACTA leak or release takes between 5-10 days for a single person.

Plans

Our nearest-term goals are the introduction of commenting on (translating/summarizing) pippies, possibly also integrating marked up texts into the fantastic co-ment.org service. Also we are going to start a dedicated blog for pippies (longstrings.soup.io), where we are going to give summaries on the results of pippifications. If you request us to process a doc, you should also be ready to write a blog-post on the results in return.

As such we also intend to integrate Pippi with Eriks other very nice tool Tratten, which does track issues from the beginning of the legislative pipeline.

If you want to support the ongoing work consider signing up and donating via . We're also participating on Mozilla's Drumbeat project project, go and vote for us.

Big plans, lot's of things to do, let's not waste time. Please submit interesting docs to longstrings on ctrlc.hu.

Thanks go to a lot of supporters: amelia for setting us up, jaywalk for hosting us, asciimoo for his coding, erik for the idea and general support, jz for useful criticism, and the telecomix guys.

Utterson is the static blogengine powering this blog. It is based on a bunch of shell-scripts and a self-generating makefile, only basic unix tools have been used (m4, gnu make, ksh93, rsync, ssh, etc). It comes with emacs integration and support for mailing in blogposts automatically via procmail. The engine has been serving this blog for the last 6 months without problems. Only a couple of small changes where necessary since the introduction of utterson, these have been commited to the github repo and are available as a v1.0 download. For more information head over to the README.

In Barcelona I showed off my timecloud widget. Then someone /* thx: a ;) */ asked if it is possible to add tagclouds to the European database of laws, eur-lex. This was the birth of bonobo.

Bonobo is a greasmonkey script which automatically retrieves the tagclouds for all legislative text available in HTML. All you need is the Greasemonkey Firefox plugin and bonobo. According to wikipedia this script might also work on other browsers than Firefox.

If you have both greasemonkey and bonobo installed, start browsing here for example.

Bonobo gets the tagcloud information from a back-end server, which retrieves, caches and converts the legislative texts to tagclouds. Like Bonobo this back-end server is also part of a set of tools trying to cut through the fog of legislation. If you're interested in the code (AGPLv3) visit the project on github.

Please bear in mind, that this back-end server is running in a quite unstable environment. It might happen, that you're not served your tagclouds. If anyone can donate some spare hosting for this service, that would be awesome. It would really feel at home on a Debian server. ;)

I have some other ideas queued already for further developments, check this project github page regularly.

Az idei 26c3 kongresszuson a legnagyobb lokális visszhangot az első nap Wikileaks előadása (stream,torrent) kapta: percekig állva tapsolták őket. Megérdemelték! Sajnos anyagilag nem állnak olyan jól, mint szimpátiával, nagy szükségük van az adományokra, hogy folytathassák a korrupció elleni harcukat. Mielőtt tovább olvasnál menj és adakozz! - És ha tudsz, futtass tor node-ot is. :)

Az idei előadáson bemutatták az elmúlt egy év sikereit és publikációit, ezzel érdemelték ki végül az ünneplést. A csattanó csak ezután következett. Bejelentették, hogy az izlandi parlamenttel egy törvényjavaslat kidolgozásán dolgoznak együtt. Amelyet még a hó végén várhatóan el fog fogadni az izlandi parlament.

Az egész ott kezdődött, hogy a wikileaks részleteket publikált arról, hogy az izlandi gazdaság összeomlása előtt pár nappal kik és hogyan mutyiztak ki 5 milliárd dollárt az országból. öt perccel az adás előtt bíróság tiltotta le az izlandi tévé híradóban a kapcsolódó tudósítás sugárzását. A TV ehelyette öt percen keresztül a wikileaks kezdőoldalát mutatták, azzal a felirattal, hogy ott minden info megtalálható. Egy háromszázezres országban ilyesminek elég nagy visszhangja van. Nemsokára már minden TV adás a wikileakses srácokat interjúvolta. Itt kihasználták a lehetőséget - never waste a crisis / sose vesztegess el egy krízist - jelszóval üzleti tervet javasoltak az izlandi gazdaság megmentésére.

Az ötlet az, hogy olyan törvényeket hoznak, amelyek más országokban már bizonyítottak. Ilyenek például a belga újságírókat, a svéd ISP jogait védő törvények, vagy az USA alkotmányának első kiegészítése. Ezzel egy fajta adat off-shore paradicsomot hoznak létre. A projekt sikeréhez bizonyos előnyös adottságok megvannak, például az energia függetlenség, az általános hűvős klíma és a politikai hangulat. Ha január végén sikerülne megszavaztatni a törvényt, akkor fontos eszközt nyerhetünk a transzparencia és az információszabadság elterjesztéséhez.

A 26c3-an felmerült, hogy a Barcelonai Charta is kiváló alapanyag lehet az izlandi törvényekhez. Az együttműködés a két csapat között már a barcelonai kiáltvány workshopján elkezdődött, szerintem hozzá tudunk tenni pár dolgot.

Ha még nem adományoztál a wikileaksnek az imént, most van még egy ok rá.

\o/ A nyissznek sikerült egy törvénymódosítással elérnie, hogy az Elektronikus közszolgáltatásokról szóló törvényben az API-k közhasznúak legyenek:

oa) amely bárki számára a terjesztés költségeinél nem magasabb ellenérték fejében vagy ingyenesen, regisztráció vagy más feltétel nélkül hozzáférhető, bárki által ingyenesen, feltétel nélkül használható,

ob) amelynek való megfelelés megvalósítását más jogalany joga, továbbá más – a közhasznúság követelményeinek megfelelő kapu követelményeinek meg nem felelő – szabvány, technikai vagy más követelmény nem korlátozza, és

oc) amelynek megfelelő termékek, szolgáltatások vagy rendszerek informatikai és kommunikációs együttműködési képességének biztosításához szükséges információk megszerzését és használatát más jogalany joga nem korlátozza.

Ha a fentieket jól értelmezem, az ilyen közhasznú kapukhoz szabad szoftveres megoldásokkal is lehet csatlakozni. Ez igen nagy eredmény. A közhasznúság ilyen újraértelmezése szerintem közel kerül a nyílt szabvány fogalmához, ám nélkülözi például a nyílt és demokratikus szabvány specifikációs és karbantartási szempontjait.

Dicséretes az munka is, ahogyan ismertetésre kerül a tv módosításhoz vezető hosszú háttérmunka és ahogyan az elért eredményeket értelmezik.

Szép munka, gratula. A igazi munka most kezdődik. :)

A tegnapi napon a La Quadrature du Net több más jónevű szervezettel együtt nyílt levelet írt az európai szerveknek az ACTA egyezménnyel kapcsolatos aggályaikkal kapcsolatban. A levél aláíróihoz a Nyílt Szabvány Szövetség is csatlakozott. Az egyezménnyel a legnagyobb baj, hogy a nyilvánosság és demokratikus ellenőrzési folyamatok kizárása mellett történnek a tárgyalások. Szerintem olyan kínosak a részletek, hogy egyetlen politikus sem vállalná most egy ilyen egyezmény nyílt támogatását.

sorry mindenkitol. sikerült megidéznünk a technika ördögét. az ATOM feedben a linkek nem müködtek. javítva. treffhetesnek hála.

ez a blog megnyílt. this blog is open.

a korábbi posztok még az régi címen elérhetőek. De az új fnord válogatás már itt jelent meg. Mindenki irányitsa át ide az rss readerjét, köszönöm.

cp -r http://yompb.wordpress.com https://www.ctrlc.hu/~stef/blog

Háttér

Az új blogmotorom az Utterson – találjátok ki, miért hívják így. Egy nagyon egyszerü blogmotorra volt szükségem, ami szépen integrálódik az emacs-omba. A felhasznált technológiák kiválásztásánál a lehető legunixosabb megoldásra törekedtem. Így egy GNU Makefile állítja elő a statikus html és xml fájlokat, korn sh szkriptekkel, m4 template motorral, kis awk-val és procmail receptekkel megspékelve.

Mivel az oldal statikus és nem akarok a spammerekkel szenvedni (ez kétségtelenül kényelmesebb a wp.com-on), kommentek nem lesznek. Ha sikerül a disqusba átimportálni a wordpress megjegyzéseket, akkor lehet. De nem igazán magas prioritású feladat ez most számomra.

Tudom elég rizikós egy ilyen költözés, egyrészt az ember olvasókat veszít, másrészt a saját infrastruktúra kezdetben még nem teljesen érett, izgalmas idők elé nézünk.

Átirányitottad az rss readeredet? Köszönöm.

És akkor alább jöjjenek a fnordok.