PHP fgetcsv() not reading all lines

Question

I have a php script that is reading a remote CSV file, and adding products to a database based on the contents of the CSV file. At present there are about 2800 lines (products) but the script keeps stopping at line 1388.

The code I used is as follows:

while(($data = fgetcsv($fopen, 0, ",")) !== false):
  //stuff is done here...
endwhile;

I have set the php memory limit to 64M and even tried 128M. I also set the max_execution_time to 60mins. I have also tried altering the code as follows:

while(($data = fgetcsv($fopen, 1000, ",", '\r')) !== false):
  //stuff is done here...
endwhile;

That DID result in more lines being parsed, BUT the data was then incorrect, i.e. image columns were becoming description columns etc. I assume that has to do with adding \r as my line ending. I tried \n, no luck. Lastly, I also added the auto_detect_line_endings as true in the ini.

Can anyone suggest reasons as to why my data is being cut short?

Regards, Simon

EDIT

I have noticed something interesting. I have a MySQL insert on each line that is looped over in the above code. Now, the last record in my database is the FIRST row in the CSV file, does this mean the file is being parsed from the last line up??

These seem to be the rows at or near the break:

W-3066,  I Love Love Cheap And Chic,     Moschino, 3.4 oz,EDT Spray,Women,,"Introduced by the design house of Moschino, I love love has a blend of grapefruit, orange, lemon, red currant, tea rose, cinnamon leaves, musk, cedar and tonka wood. It is recommended for daytime wear.",http://www.perfume-worldwide.com/products/Women/Final/W-3066large.jpg,0,0,0,8011003991457
W-3070,  Adidas Floral Dream,            Adidas,   1.7 oz,EDT Spray,Women,,"Introduced in 2008, the notes are bergamot, lily, rose, tonka bean and vanilla.",http://www.perfume-worldwide.com/products/Women/Final/W-3070large.jpg,0,0,0,3412244310024
W-3071,  Adidas Fruity Rhythm,           Adidas,   1.7 oz,EDT Spray,Women,,"Introduced in 2008, the notes are black currant, raspberry, cyclamen, freesia and musk.",http://www.perfume-worldwide.com/products/Women/Final/W-3071large.jpg,0,0,0,3412244510004

---

SOLUTION

As it turns out, it worked out a lot better for me to copy the file to my server, and work off the copy. The steps I followed are as follows:

• I read the contents of the remote file using file_get_contents()
• I then used iconv() function to re-encode data to UTF-8
• I made a temp file using fopen(), fwrite() and fclose() functions, contents of the file was the encoded data above
• I set the permissions of the file to 0750 using the chmod() function
• I then applied the fgetcsv() function to my temp file
• Did all that needed to be done
• Deleted the temp file once done, using unlink() function

That did the trick. So, I suspect half the issue was actually the remote server timing out, and the other half encoding issues.

Thank you to everyone for all the nudges in the right direction

Answer 1

is the file correctly formated? have you tried to open the file it in some csv reader in which you can specify delimiters and end lines)? Judging by this:

That DID result in more lines being parsed, BUT the data was then incorrect, i.e. image columns were becoming description columns etc

I would assume that data maybe is corrupted (i.e. some description had comma, endline, etc) It happneds if data is generated dynamically and not formatted correctly.

open in txt editor as well (i.e notepad++) and see how that goes/looks..