Initial database population from large data files, several problems

by Justin Allen Jaynes » Fri, 06 Mar 2009 05:13:56 GMT


Sponsored Links
 I'm building a dictionary application with 135,000 word entries (words 
only).  My raw file must have been too large (1.5 meg), because I got 
this error:

D/asset (909): Data exceeds UNCOMPRESS_DATA_MAX (1424000 vs 1048576)

I've searched for this error with very few relevant hits.  It seemed to 
mean I could not open an uncompressed file over a meg.  So I then split 
the file into two smaller files and ran my code on both of them.  It 
worked out fine.  My total application size is 3 meg installed.

My code is:
public void onCreate(SQLiteDatabase database) {
    database.execSQL("CREATE TABLE " + DATABASE_TABLE + " (wordid 
INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, word VARCHAR);");
           
    Scanner fileScanner = new 
Scanner(myContext.getResources().openRawResource(R.raw.wordlist));
    while ( fileScanner.hasNextLine() ) {
        String wordFromFile = fileScanner.nextLine();
         database.execSQL("INSERT INTO words (word) VALUES ('" + 
wordFromFile + "');");
    }
    fileScanner = new 
Scanner(myContext.getResources().openRawResource(R.raw.wordlist2));
    while ( fileScanner.hasNextLine() ) {
        String wordFromFile = fileScanner.nextLine();
         database.execSQL("INSERT INTO words (word) VALUES ('" + 
wordFromFile + "');");
     }
}

However, when the application is first run, it takes several MINUTES to 
initialize the database in this way.  Is there a way (like a copy 
command, as found in, say, postgresql, or a restore of a database file) 
to copy data from a raw file, and can such a method be accessed from the 
SDK so that standard first-run procedures can correctly set up the 
database?  I have been unable to locate such a luxury.  I am seeking to 
speed up this data populating process.

First question: how can I speed up my database population?

Second question: is there a way to read a raw resource file larger than 
1 megabyte (aside from making it into two smaller files)?  If not, why?

Justin

--~--~---------~--~----~------------~-------~--~----~



Other Threads

1. OpenIntents 0.1.5 released - with Media Player

The latest release of OpenIntents features a new Media player that
lets you play your favorite songs and videos from internal memory as
well as from the SD card. Building on top of the media player is
SplashPlay that lets you easily learn how to play the guitar.

We have enhanced the Content browser by a MultiWordAutoComplete
feature and a context menu, and added new map functionality to
Locations.

Our Newsreader now also supports Atom feeds and the Newsreader service
can start on boot if desired.

Get the binary or source code release 0.1.5 from our download page at
our new homepage:

http://www.openintents.org/en/

We look forward to your feedback!

Peli

--~--~---------~--~----~------------~-------~--~----~

2. Disable Log for Release

How can iam disable the Log outputs for the release-apk?

lars


--~--~---------~--~----~------------~-------~--~----~

3. Sms to a number

4. suggestion: the *relevant* context should be automatically accessible from any method

5. How to change the color of the title bar?

6. Running (Ordering) code just after SearchComplete() function is finished

7. Idea for Android App up for Grabs - Use it if you want it.