Eriky.com

ESClient: Python ElasticSearch Client

Erik-Jan — Sat, 25 Feb 2012 20:23:55 +0000

I’ve been working on a little project: a Python client for ElasticSearch. Are there other clients out there? Yeah sure, and they are even pretty decent. Especially pyes is OK. But I’m missing documentation on that one and I don’t like the approach they took on implementing the API. I wanted to create a simple client that stays close to the ElasticSearch REST API. It allows you to directly submit either JSON or a tree of Python objects that can be converted to JSON. ESClient has documented code and supplied unittests that should get you a long way. I plan to write good documentation to get people started quickly as soon as the API starts to reach a stable state.

In a few days I have learned a lot about:

Github
Packaging Python Libraries
publishing a package to the Python Package Index.
Semantic Versioning

It’s simply awesome how quick and easy you can get something up and running these days. With just a few lines of code and Pythons distutils you can create a source package and a Windows binary, upload it all to PyPI and make it installable for everyone in the world with a single command:

pip install esclient
or:
easy_install esclient

ESClient is still in its early stages and I can assure you the API will change over the coming weeks. I need to implement more API methods and I want to implement bulk indexing. If all API methods are implemented and bulk indexing works I will work towards a stable 1.0.0 release that will also have a stable API. I also want ESClient to handle errors well. This is something that I missed in some other libraries that I have found, like pyelasticsearch.

Install Canon software without the original CD

Erik-Jan — Sun, 28 Aug 2011 14:27:58 +0000

Canon has an annoying and useless policy which makes it impossible to install the updates from their website without having an original cd. Apparently they want to make the lives of their customers harder than it already is. (My Mackbook Air doesn’t even have a DVD player!)

Lucky for us this website has some tricks to circumvent this. For me simply removing an update.plist file was enough to get ImageBrowser 6.7.2-updater (Mac OS 10.5-10.6) to installing without original cd.

Twitter’s list of 401 banned passwords

Erik-Jan — Fri, 19 Aug 2011 20:22:08 +0000

I was looking at the source of a Twitter.com page when I noticed a javascript array with banned passwords. They were ROT13 encoded, so I decided to decode them and have a look. This has been done before but the list is now a bit longer, so here you go:

(warning: this list contains quite some dirty words, which could be the reason why Twitter does a rot13 enconding :-0)

1: zzzzzz
2: zxcvbnm
3: zxcvbn
4: yellow
5: yankees
6: yankee
7: yamaha
8: xxxxxxxx
9: xxxxxx
10: xavier
11: wizard
12: winter
13: winston
14: winner
15: wilson
16: willie
17: william
18: whatever
19: welcome
20: warrior
21: walter
22: voyager
23: voodoo
24: viking
25: victoria
26: victor
27: vagina
28: united
29: twitter
30: turtle
31: tucker
32: trustno1
33: trouble
34: travis
35: toyota
36: topgun
37: tomcat
38: tigger
39: tigers
40: tiffany
41: thx1138
42: thunder
43: thomas
44: theman
45: testing
46: tester
47: teresa
48: tennis
49: taylor
50: tequiero
51: sydney
52: swimming
53: surfer
54: superman
55: sunshine
56: summer
57: suckit
58: success
59: stupid
60: sticky
61: steven
62: steelers
63: starwars
64: startrek
65: srinivas
66: squirt
67: spider
68: sparky
69: spanky
70: sophie
71: soccer
72: snoopy
73: smokey
74: slayer
75: skippy
76: silver
77: sierra
78: shaved
79: shannon
80: shadow
81: sexsex
82: secret
83: sebastian
84: scorpion
85: scorpio
86: scooter
87: scooby
88: saturn
89: sandra
90: samson
91: sammy
92: samantha
93: russia
94: rush2112
95: runner
96: rosebud
97: rocket
98: roberto
99: robert
100: richard
101: redwings
102: redsox
103: redskins
104: rebecca
105: rangers
106: ranger
107: rainbow
108: raiders
109: racing
110: rachel
111: rabbit
112: qwertyui
113: qwerty
114: qazwsx
115: pussies
116: purple
117: private
118: princess
119: prince
120: porsche
121: pookie
122: please
123: player
124: phoenix
125: phantom
126: pepper
127: peanut
128: peaches
129: patrick
130: password123
131: password12
132: password1
133: password
134: password
135: parker
136: panties
137: panther
138: packers
139: orange
140: oliver
141: nipples
142: nipple
143: nicole
144: nicholas
145: newyork
146: ncc1701
147: naughty
148: nathan
149: nascar
150: naked
151: mustang
152: murphy
153: muffin
154: mountain
155: mother
156: morgan
157: monster
158: monkey
159: monkey
160: monica
161: mistress
162: miller
163: midnight
164: mickey
165: michelle
166: michael
167: merlin
168: mercedes
169: member
170: melissa
171: maxwell
172: maverick
173: matthew
174: matrix
175: master
176: marvin
177: martin
178: marlboro
179: mariposa
180: marine
181: magnum
182: maggie
183: madison
184: maddog
185: lovers
186: london
187: little
188: letmein
189: letmein
190: legend
191: leather
192: lauren
193: lakers
194: ladies
195: knight
196: killer
197: justin
198: junior
199: joshua
200: joseph
201: jordan
202: johnson
203: johnny
204: jessica
205: jeremy
206: jennifer
207: jasper
208: jasmine
209: jaguar
210: jackson
211: jackie
212: iwantu
213: internet
214: iloveyou
215: iceman
216: hunting
217: hunter
218: hotdog
219: horney
220: hooters
221: hockey
222: hentai
223: helpme
224: heather
225: harley
226: hardcore
227: hannah
228: hammer
229: gunner
230: guitar
231: gregory
232: gordon
233: golfer
234: golden
235: gizmodo
236: ginger
237: giants
238: george
239: gemini
240: gators
241: gateway
242: gandalf
243: fuckyou
244: fuckme
245: fucking
246: fucker
247: fucked
248: freedom
249: freddy
250: forever
251: football
252: flyers
253: flower
254: florida
255: fishing
256: firebird
257: ferrari
258: fender
259: falcon
260: extreme
261: estrella
262: erotic
263: einstein
264: edward
265: eagles
266: eagle1
267: driver
268: dreams
269: dragon
270: donald
271: dolphins
272: dolphin
273: doggie
274: doctor
275: diamond
276: diablo
277: dennis
278: debbie
279: danielle
280: daniel
281: dallas
282: dakota
283: cumshot
284: cumming
285: crystal
286: cowboys
287: cowboy
288: corvette
289: cooper
290: cookie
291: consumer
292: computer
293: compaq
294: college
295: coffee
296: cocacola
297: chicken
298: chicago
299: chester
300: chelsea
301: cheese
302: charlie
303: charles
304: casper
305: carter
306: carlos
307: captain
308: canada
309: cameron
310: camaro
311: calvin
312: butthead
313: butter
314: buster
315: bulldog
316: broncos
317: bronco
318: brazil
319: braves
320: brandy
321: brandon
322: boston
323: boomer
324: booger
325: booboo
326: bonnie
327: bonita
328: bond007
329: blowme
330: blowjob
331: blondes
332: blonde
333: blazer
334: biteme
335: bitches
336: birdie
337: bigtits
338: bigdog
339: bigdick
340: bigdaddy
341: bigcock
342: beavis
343: beaver
344: beatriz
345: batman
346: baseball
347: barney
348: banana
349: bailey
350: badboy
351: austin
352: august
353: asshole
354: ashley
355: asdfgh
356: asdfgh
357: arthur
358: arsenal
359: apples
360: apollo
361: anthony
362: animal
363: angels
364: angela
365: andrew
366: andrea
367: america
368: amateur
369: amanda
370: alejandro
371: alejandra
372: alexis
373: alberto
374: albert
375: action
376: access14
377: access
378: abgrtyu
379: abcdef
380: abc123
381: abc123
382: aaaaaa
383: 987654
384: 8675309
385: 7777777
386: 777777
387: 696969
388: 666666
389: 654321
390: 232323
391: 131313
392: 123456789
393: 12345678
394: 1234567
395: 123456
396: 123123
397: 121212
398: 112233
399: 11111111
400: 111111
401: 000000

Macbook Air fan stuck and making ticking noise?

Erik-Jan — Thu, 04 Aug 2011 08:33:15 +0000

Does your Macbook Air fan makes a repeating, predictable ticking noise? Does it get hotter than usual when watching YouTube video’s? Does iStat Pro reports a fan speed of 0?

A few taps with your finger help! If you have an older model (2009) tick on the right side. If you have a newer model (mine is late 2010) then tap on the right side. Don’t overdo it, but don’t be too gentle either.

See also this discussion: https://discussions.apple.com/thread/1860264

Want to beat the hackers at their own game?

Erik-Jan — Fri, 07 May 2010 19:10:23 +0000

Google shows how web application vulnerabilities can be exploited and how to defend against these attacks. The best way to learn things is by doing, so you’ll get a chance to do some real penetration testing, actually exploiting a real application.

http://jarlsberg.appspot.com/

Anti Counterfeiting Treaty Agreement leaked

Erik-Jan — Sat, 27 Mar 2010 11:01:44 +0000

The leaked ACTA agreement!
Soon to be copy pasted into a law near you.

Check out this page on The piratebay

Another privacy invading monster (LSO cookies)

Erik-Jan — Tue, 18 Aug 2009 08:36:04 +0000

Recently I came across something called LSO cookies (read about it, I’m not going to explain them in detail here). Since more and more browsers, virus scanners and security software block cookies, these LSO cookies are a real treat for advertisers and tracking companies for several reason:

most people never heard of them
they are difficult to block
they provide lots of storage (100KB per site)
they are not removed by your browser, ever
they work and track you, even with your browsers “privacy mode”

Dangers

There are several dangers to these cookies. First and foremost, we block cookies for a reason. We don’t want to be tracked everywhere on the web and we don’t want companies to build profiles of our web usage for whatever reasons they have. These companies shamelessly track us anyway by using all kinds of tricks, like these LSO cookies, instead of respecting our explicit choice to not be tracked and monitored.

Another problem is that this will leave tracks of your Internet usage on your computer, even if you try to cover those tracks by deleting cookies, browser cache and temporary files.

So what can we do about this?

First of all, the best thing would be to not use flash but that ain’t an option. (We want our youtube to work!). So the second best option is to block or at least remove the cookies. There is an excellent Firefox plugin called Better Privacy that will give you all kinds of options to remove or block LSO cookies.

If you don’t have Firefox, your third option is to go to Abobe’s Flash player settings page – you never heard of it, neither did I – and set the storage space to zero KB. Next, go to the last tab there or use this link, and be amazed at the amount of sites that use LSO cookies to store whatever they want to store on your PC. Next, click the remove all button to remove it all. Note that setting the storage to zero prevents sites from storing cookies, but Flash will still create directories for each site that tries. So next time you visit that shameful pr0n site, be aware that Flash will keep track of it.

WikiBench master thesis

Erik-Jan — Wed, 20 May 2009 08:24:46 +0000

I officially got my WikiBench project graded with an 8, with which I’m of course very satisfied. You can now read my thesis called WikiBench: A distributed, Wikipedia based web application benchmark.

People interested in this project already found their way to my blog. For those who are wondering: I will publish the code and I will do so very shortly (within days). It will most probably appear on Google Code and you won’t have to search for it since I will devote a post to it right here and include the URL. I will probably release it under a BSD-style license which should give you lots of freedom. Unfortunately I’m not sure yet if I am allowed to release some of the trace files obtained from Wikipedia.

Ghostnet

Erik-Jan — Tue, 14 Apr 2009 09:19:17 +0000

After all the buzz around Ghostnet, it’s fun to look back and read the origal document describing the spy network. It’s an interesting read, and if you don’t have the time to read this you can also check out the Security Now! podcast from April 9th in which Steve Gibson explains how the research group found out about the spy network and how amateurish the (open source) Gh0trat software actually is.

One very important lesson learned from this story is that attackers no longer control these networks by using IRC as we have seen in the past. Ghostnet used plain old http requests to periodically check for new commands. The startling thing about this is that this is exactly the kind of traffic that gets through firewalls and even proxy servers without any problems. HTTP replies consisting of jpg images contained the actual, encoded commands.

Bloom filters

Erik-Jan — Tue, 03 Mar 2009 22:19:11 +0000

This article by Broder and Mitzenmacher gives a good description of how bloom filters work and what they can do for you. The bloom filter basically replaces a dataset with a filter that can tell you if an item is a member of that set or not. It will not give false negatives, but it might give false positives. In practise, this is a negative property that can be outweighted by the space savings a bloom filter introduces; after all, you do not need to query the dataset to determine membership. The most important and summarizing quote you should remember from the article:

The Bloom ﬁlter principle: Wherever a list or set is used, and space is at a premium, consider using a Bloom ﬁlter if the effect of false positives can be mitigated.

The article also gives a number of examples in which bloom filters are used. E.g. to aid resource location in P2P and cache systems.