Comments on: Importing the complete wikipedia database in 5 hours http://www.eriky.com/2008/11/importing-the-complete-english-wikipedia-database Just another geek with a blog Mon, 02 Aug 2010 19:20:22 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: Erik-Jan http://www.eriky.com/2008/11/importing-the-complete-english-wikipedia-database/comment-page-1#comment-120 Erik-Jan Mon, 02 Aug 2010 19:20:22 +0000 http://www.eriky.com/?p=3#comment-120 Sorry for the late reply, but yes this can take very long. The more memory you put in your pc, the faster the indexing goes because swapping is the main cause I think. Sorry for the late reply, but yes this can take very long. The more memory you put in your pc, the faster the indexing goes because swapping is the main cause I think.

]]> By: GeBoZhuang http://www.eriky.com/2008/11/importing-the-complete-english-wikipedia-database/comment-page-1#comment-117 GeBoZhuang Mon, 28 Jun 2010 14:04:37 +0000 http://www.eriky.com/?p=3#comment-117 Hi Erik, I'm also trying to import wikipedia using Plan B:) But after many tries, there is some problem remained and I wanna to know whether you had encountered it. After DROP PRIMARY KEY, DROP INDEX and MODIFY (auto_increment column) on tables "page", "revision" and "text", the importing speed is indeed very fast(about 2000/sec). But when the importing is finished and I want to ADD the primary key and the indexes, the speed is very slow, especialy on table "text". It taking more than 4 hours before I canceled it, since I don't know whether there is some problems. On your computer, how much time did it take to recreate the indexes after importing ? Thank you :) Hi Erik, I’m also trying to import wikipedia using Plan B:) But after many tries, there is some problem remained and I wanna to know whether you had encountered it.

After DROP PRIMARY KEY, DROP INDEX and MODIFY (auto_increment column) on tables “page”, “revision” and “text”, the importing speed is indeed very fast(about 2000/sec). But when the importing is finished and I want to ADD the primary key and the indexes, the speed is very slow, especialy on table “text”. It taking more than 4 hours before I canceled it, since I don’t know whether there is some problems.

On your computer, how much time did it take to recreate the indexes after importing ? Thank you :)

]]>