Initial database population from large data files, several problems

by Justin Allen Jaynes » Fri, 06 Mar 2009 05:13:56 GMT


Sponsored Links
 I'm building a dictionary application with 135,000 word entries (words 
only).  My raw file must have been too large (1.5 meg), because I got 
this error:

D/asset (909): Data exceeds UNCOMPRESS_DATA_MAX (1424000 vs 1048576)

I've searched for this error with very few relevant hits.  It seemed to 
mean I could not open an uncompressed file over a meg.  So I then split 
the file into two smaller files and ran my code on both of them.  It 
worked out fine.  My total application size is 3 meg installed.

My code is:
public void onCreate(SQLiteDatabase database) {
    database.execSQL("CREATE TABLE " + DATABASE_TABLE + " (wordid 
INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, word VARCHAR);");
           
    Scanner fileScanner = new 
Scanner(myContext.getResources().openRawResource(R.raw.wordlist));
    while ( fileScanner.hasNextLine() ) {
        String wordFromFile = fileScanner.nextLine();
         database.execSQL("INSERT INTO words (word) VALUES ('" + 
wordFromFile + "');");
    }
    fileScanner = new 
Scanner(myContext.getResources().openRawResource(R.raw.wordlist2));
    while ( fileScanner.hasNextLine() ) {
        String wordFromFile = fileScanner.nextLine();
         database.execSQL("INSERT INTO words (word) VALUES ('" + 
wordFromFile + "');");
     }
}

However, when the application is first run, it takes several MINUTES to 
initialize the database in this way.  Is there a way (like a copy 
command, as found in, say, postgresql, or a restore of a database file) 
to copy data from a raw file, and can such a method be accessed from the 
SDK so that standard first-run procedures can correctly set up the 
database?  I have been unable to locate such a luxury.  I am seeking to 
speed up this data populating process.

First question: how can I speed up my database population?

Second question: is there a way to read a raw resource file larger than 
1 megabyte (aside from making it into two smaller files)?  If not, why?

Justin

--~--~---------~--~----~------------~-------~--~----~



Other Threads

1. noHistory="true" rocks

<activity android:name=".activities.Result"
android:label="@string/arl_results"
android:screenOrientation="portrait" android:noHistory="true"></
activity>

this was invaluable in debugging Droid out of Bitmap memory issues.

-- 

2. How to select/install a locale other than English

My application has resources files in English and French and works on
Android 1.5 and >.


Inside the emulator, in Settings > Locale & Text > Select Locale, I
can choose English or French. This updates the language of my
application.

However, when I use the Android Dev Phone 1, only English is available
in the "Select Locale" screen...


Is it possible to install french locale on Android Dev Phone 1 ?
If so, how can I do this ?

Thanks for your help !

-- 

3. DynamicLayout and StringBuilder

4. Save ListView items for next time

5. How can I call service methods from broadcastReceiver?

6. Application developed in 1.5 r 3 not working in Motorola Droid

7. Bitmap questions