Differences

This shows you the differences between two versions of the page.

--- projects:digikey_partsdb [2013/10/12 10:42] – charliex
+++ projects:digikey_partsdb [2013/10/13 10:11] (current) – charliex
@@ Line 888: / Line 888: @@
 curl with a valid useragent i used --useragent "Chrome/1.0" but vary it to avoid rate limiters.
+<code>
+curl.exe -o page%1.csv -L -v -G "http://www.digikey.com/product-search/download.csv?FV=fff40008%2Cfff801b9&mnonly=0&newproducts=0&ColumnSort=0&page=%1&stock=0&pbfree=0&rohs=0&quantity=0&ptm=0&fid=0&pageSize=500" --digest --user-agent "Chrome/1.0"
+</code>
 The response has 4 bytes at the front we don't want, so a simple byteskip script or piece of code.
@@ Line 989: / Line 995: @@
 }
 </code>
+Process all the files.
+<code>
+for %a in (*.csv) do byteskip %a o%a 4
+</code>
+I used one of the online CSV to MYSQL converters, but most of them can't handle the variations in CSV. To create the initial schema for each table i converted one CSV to XLS by importing it into google docs, and then re-exporting it as an XLS then importing that into phpmyadmin, that makes the base schema.<br>
+Rename the table in phpmyadmin or via mysql tool
+Then do the final import with the csvtosql tool, (in progress)