Comments on: Mapping Wikipedia

By: links for 2008-04-29 at they made me do it

links for 2008-04-29 at they made me do it — Tue, 29 Apr 2008 04:34:17 +0000

[...] Mapping Wikipedia (tags: python wikipedia) [...]

By: joe

joe — Tue, 29 Apr 2008 00:10:22 +0000

Mapping of Wikipedia is being undertaken by Katy Borner’s network visualization group at Indiana University. The graphics are very interesting. See

http://scimaps.org/maps/wikipedia/

By: earobinson

earobinson — Mon, 28 Apr 2008 20:37:41 +0000

Sorry to Mark and H3g3m0n, not sure why your comments got marked as spam, I will look into doing that in the next version.

If anyone else had comments that are missing please contact me.

By: earobinson

earobinson — Mon, 28 Apr 2008 15:12:59 +0000

Thanks for all the comments. I was really just playing with it on my own not really worried or not if there was all ready a tool out there that did it because I just wanted to play with the code, and having the code is always a good thing and now you can play with the code to.

I love the work of Chris Harrison, and think he is the one that gave me the idea to map the internet in the first place.

@Bulkan Evcimen Ya thanks for the feedback, graphviz is currently the real benchmark so I need to figure out how to replace that or to make it a lot faster

By: Marc

Marc — Mon, 28 Apr 2008 14:16:58 +0000

Wikipedia asks that you not use a webspider to access the database. You may download the entire database of any wikimedia project from the following link:

http://en.wikipedia.org/wiki/Wikipedia:Database_download

By: Pete Skomoroch

Pete Skomoroch — Mon, 28 Apr 2008 13:20:17 +0000

Check out Matt Biddulph’s post at HackDiary:

“Using Wikipedia and the Yahoo API to give structure to flat lists”

http://www.hackdiary.com/archives/000070.html

By: H3g3m0n

H3g3m0n — Mon, 28 Apr 2008 11:29:40 +0000

Rather than use the api, you can get the download version of Wikipedia, it would be a lot faster to generate large maps from and save Wikimedia on bandwidth ☺

http://en.wikipedia.org/wiki/Wikipedia:Database_download ☢

By: Ben

Ben — Mon, 28 Apr 2008 11:12:44 +0000

http://chrisharrison.net/projects/wikiviz/index.html

is a pretty cool visualization in the same vein. Graphviz is really cool software and the results from your method are probably much more usable than Wikiviz. Good stuff.

By: jiu

jiu — Mon, 28 Apr 2008 08:17:35 +0000

Great post! it’s cool and creative.

By: Bulkan Evcimen

Bulkan Evcimen — Mon, 28 Apr 2008 07:19:27 +0000

So what would it take to create a map of all of Wikipedia?

To speed up the bit were you download the wiki entries you could use threads. So, within the thread(s) you would download the html page.

The third todo can be done in two ways, were you says ignore cache (download urls that have been seen) or not to ignore it.