Data Mining

Just wrote a little Java program to go out and fetch data and add it to my database.  It's mainly movie data right now.  Hopefully by next week, this little program will go out and fetch book and other product data.

The little data mining program simply fetches URLs that return a JSON string.  It then parses the JSON object for keywords.  Any images are downloaded and any unexpected data is thrown into "metadata."  Hopefully my online database doesn't run out of room.  Otherwise I'm going to have to add an overflow database to my private little server.  But I'm guessing Cox wouldn't like the traffic; I'd probably have to buy their business plan. Yuck.  That's a two year commitment (I'm guessing they'd have to lay out additional landlines or something).

I'm sure I could also purchase an additional MySQL database some where.  They aren't too expensive.

Well, the data mining program is only at 0.1%.  I really should have made it multithreaded.  But then again I don't want my IP banned.  Oh well.