metaprogramming and politics

Decentralize. Take the red pill.

Wikileaks or: Welcome to the brave new “Terms of Use” era

with 8 comments

Paypal “restricts” the bank account of the german Wau Holland foundation which managed parts of Wikileaks transactions. Citing from their statement :

PayPal has permanently restricted the account used by WikiLeaks due to
a violation of the PayPal Acceptable Use Policy, which states that
our payment service cannot be used for any activities that
encourage, promote, facilitate or instruct others to engage in
illegal activity. We’ve notified the account holder of this action.

As with my yesterday notes a number of questions arise. What exactly is
illegal about wikileaks behaviour in the US? Was money laundering or
online fraud involved? Was wikileaks given a warning and asked to
provide clarifying statements regarding the usage of _their_ money? Do
they apply the same moral standards to all their customers and accounts?
For example, are they also closing all accounts related to the
Washington Times or related to Sarah Palin publically calling for
assasination
of a citizen of Australia?

If we are learning anything from the Wikileaks case it is the willfullness
of young IT companies in the US to concur with public opinion or direct
government pressure. Up until now i presumed the likes of Amazon or Paypal
also incorporated some ethics. And this would mandate IMO to keep
distance to government actions and to resist government pressure especially
if it’s about something like wikileaks. It seems like the US goverment
used its advanced notice of the pending publication wisely: rather than filtering critical
messages they organised a huge campaign in the US resulting in Amazon,
PayPal and everydns and probably others to implement government policies
and quickly. No need to have special laws or evil government actions – just reference the Terms of Use and be done.

Advertisements

Written by holger krekel

December 4, 2010 at 3:11 pm

Posted in politics

Tagged with

Who needs censorship if you have Amazon and everydns?

with 13 comments

Yesterday Amazon stopped services for the wikileaks archive. Citing from the Amazon statement:

It is not credible that the extraordinary volume of 250,000 classified documents that WikiLeaks is publishing could have been carefully redacted in such a way as to ensure that they weren’t putting innocent people in jeopardy.

So here you have a popular cloud services provider judging their customers content in a broad manner, stating conclusion, terminate services, done. Did they allow discussion? Do journalists need to keep and control all rights of the material they are publishing? Can Amazon show any evidenceof their “jeopardy” conclusion? Would they apply the same moral standard to e.g. blogs or other (journalistic) content that called for going to the war on Iraq – causing >100.000 civilians to die? Does Amazon really want us to believe the US governments outrage and Mr. Liebermanns actions are merely a co-incidence?

Then today everydns.net terminated DNS services for wikileaks.org with a 24 hour prior notice. Citing from the everydns statement:

More specifically, the services were terminated for violation of the provision which states that “Member shall not interfere with another Member’s use and enjoyment of the Service or another entity’s use and enjoyment of similar services.”

So supposedly there was an attack on their DNS servers and they interpreted this as wikileaks doing harm to others? Is this the future of how we handle attacks against single domain names? Would the same happen if it was the site of a chinese dissident or a Poker playing site or any other customer?

As things stand this puts me off these two companies and brings me again to the thought that we need an internet that is as independent from any single company or any single country as possible – it’s in the best interest for all of us in the long run.

Written by holger krekel

December 3, 2010 at 12:44 pm

Posted in politics

Tagged with

Ring of Python talk at pycon

with 3 comments

Just did my talk on Ring of Python talk at Pycon US 2010, discussing competition and features of Python interpreters and co-operation issues around the most important issue in my oppinion: deployment. Also showcased execnet as a generic Python2Python bridge, connecting Python2.4, Python 3.1, Jython and IronPython. Got some nice feedback, also about the presentation style, actually i was using Prezi. You may go to the following page, click into the flash app and hit “cursor right”: Ring of Python .

Written by holger krekel

February 19, 2010 at 11:06 pm

Posted in metaprogramming

Elastic Python deployment networks

with 2 comments

Time for a bit of fiction on distributed Python deployment. As some of you know, py.execnet imperatively and elastically executes code in local or remote python processes, maintaining channels for exchanging data. Execnet has the wonderful zero-install feature which means no software except a Python interpreter is required remotely. The connection between Python interpreters is direct, i.e. the connecting side needs to know how to start the remote side. And it’s non-transitive meaning: given a A->B and a B->C connection there is no support for getting a A->C connection mediated by B.

I’d like to lift these restriction and introduce the concept of a deployment network through which execnet connections can be mediated. I am pondering a command line tool that creates a network of Python intepreters on multiple hosts like this:

execnet start mynet ssh=linuxbox.org socket=windowsbox.com ssh=osx.com

This would create a "mynet" deployment network of four Python interpreters running on different hosts and platforms: one local process and three remote processes connected to it. Let’s add a remote Jython process to the "mynet" deployment network:

execnet addhost mynet ssh=remote//python=jython

We can generally use the ‘mynet’ handle to work with this newly instantiated deployment network. For example, to get a fresh process on a certain platform from a Python program:

mynet = execnet.connect('mynet')
gateway = mynet.makegateway(platform="java")

The first line connects to our local ‘mynet’ process. The second line creates a gateway to a fresh Python interpreter, in this case a Jython process. The bootstrapping of the Jython-side gateway object is determined by the initiating client side. The two subprocesses communicate through an IO-connection that is mediated by the ‘mynet’ deployment network.

This is very exciting because the zero-installation feature is preserved on two levels: the deployment processes work on software coming from a single point, the command line above. And our "on-top" gateways operate with software determined from the initiating side, from the python code above. Interaction between the two worlds is limited to a connect operation and providing IO mediatiation. This means the deployment network facilities can evolve independently from the "on-top" execnet-elastic programs.

Conceptually it’s a very reliable and robust setting. The mynet processes should be able to run as robustly as unix shells. They are to provide a solid base for writing and deploying Python applications that span multiple interpreters. They don’t run applications in-process.

This is not all fiction. The current development version of py.execnet already works across and between CPython2.4 through to CPython 3.1, Jython and PyPy. And i intend to release execnet as a separate package soon, providing the basis for implementing the above fiction and lots of other on-top fun 🙂

Written by holger krekel

September 26, 2009 at 9:29 pm

Posted in metaprogramming

Let’s prevent a no-privacy world by better technology!

with one comment

Dear Google, Amazon and web 2.0: you are doing a great job of providing cool services and apps, you are doing away with obnoxious installation and upgrade steps, your stuff often works out of the box and you offer convenient interfaces. Great.

Comes with a caveat, though: the loss of control of my data and my communication, loss of ability to install and run whichever programs I like. You as central organisations and your ruling governments develop the practical possibility to get at all this data, on a mass basis and retro-actively. Something needs to change about this or we all will be ending in a world with historically unprecedented power structures in the hand of few. A world with virtually no privacy.

Part of what i can do is thinking about cool new technology to counter these developments. I am convinced we need a more decentral application execution infrastructure. We need open cloud software and infrastructure that allows to have easy-to-develop apps run "on the net" – a wealthy network of PCs and mobile phones. A free wireless-type network, not expensive mobile "total control" networks.

Written by holger krekel

August 15, 2009 at 1:40 pm

pylib 1.0.0 released: the testing-with-python innovations continue

with 5 comments

Took a few betas but finally i uploaded a 1.0.0 py lib release, featuring the mature and powerful py.test tool and "execnet-style" elastic distributed programming. With the new release, there are many new advanced automated testing features – here is a quick summary:

  • funcargs – pythonic zero-boilerplate fixtures for Python test functions :
    • totally separates test code, test configuration and test setup
    • ideal for integration and functional tests
    • allows for flexible and natural test parametrization schemes
  • new plugin architecture, allowing easy-to-write project-specific and cross-project single-file plugins. The most notable new external plugin is oejskit which naturally enables running and reporting of javascript-unittests in real-life browsers.
  • many new features done in easy-to-improve default plugins, highlights:
    • xfail: mark tests as "expected to fail" and report separately.
    • pastebin: automatically send tracebacks to pocoo paste service
    • capture: flexibly capture stdout/stderr of subprocesses, per-test …
    • monkeypatch: safely monkeypatch modules/classes from within tests
    • unittest: run and integrate traditional unittest.py tests
    • figleaf: generate html coverage reports with the figleaf module
    • resultlog: generate buildbot-friendly reporting output
  • distributed testing and elastic distributed execution:
    • new unified "TX" URL scheme for specifying remote processes
    • new distribution modes "–dist=each" and "–dist=load"
    • new sync/async ways to handle 1:N communication
    • improved documentation

The py lib continues to offer most of the functionality used by the testing tool in independent namespaces.

Some non-test related code, notably greenlets/co-routines and api-generation now live as their own projects which simplifies the installation procedure because no C-Extensions are required anymore.

The whole package should work well with Linux, Win32 and OSX, on Python 2.3, 2.4, 2.5 and 2.6. (Expect Python3 compatibility soon!)

For more info, see the py.test and py lib documentation:

http://pytest.org

http://pylib.org

have fun, holger

Written by holger krekel

August 4, 2009 at 10:05 am

code-centered issue tracking?

with 12 comments

Is there anything that allows code-centered issue tracking? Recently, Gustavo Niemeyer had an interesting piece up where he suggests private/protected syntax for Python. His point is that code collaboration doesn’t otherwise work in larger uncontrolled environments of dev groups. I agree with most of his observations but not his conclusion. I’d rather like to see reduced communication costs for changing code. Here is an example of what i mean. If see a code fragment like this:


def somefunc(self, x, y, z):
    self._cache = func(x) + other(y)
    self.z = self._cache + third(z)

i want to be able to visually mark this code, write a comment like:

hey, i need the third(z) value, and doing self.z-self._cache feels bad – can you help?

and i want my development environment to automatically route this question with exact code refs to the maintainers of the code. This should not take longer than 20 seconds and be automatically managed.

On the receiving side, as the maintainer, i want to get notified and be able to say:

getissues mypkg/subpkg

and have it automatically list me all files and issues for it. So i easily see the above question, do a patch, and issue:

sendpatch PATCHNAME

and type in a message. Maybe automatically CCed to other library maintainers, a mailing list etc. As the original sender i get back a mail and can use a cmdline tool to apply the patch, give feedback and so forth …

IOW, i want to have tools that automatically manage the issue-addressing, code-referencing, finding-out about package info/release numbers, format the comment related to the cited code, send the mail, register an issue automatically and so on. Is there something like this?

If not: doesn’t we already have most of the pieces? What makes sense to use for it? Mercurial and Patch queues? Maybe the new Bitbucket API? Integrate with existing issue tracker? First goal would be to have it manage itself, i guess 🙂

cheers, holger

Written by holger krekel

May 18, 2009 at 11:35 am

Posted in metaprogramming