Spreadsheet import fails with an SQL error when a text field contains a question mark

Submitted by John K on

I hit a reproducible import failure today and tracked down the cause, so I am writing it up in case it is useful.

If any text value in an import file contains a question mark, the import fails with a server error and the downloadable error file is empty.

I uploaded a 935-row .xlsx of moth records in training mode. Column mapping and species value-matching both completed normally. On starting the import:

▎ "An error occurred on the server whilst importing your data."

The errors file listed all 935 rows with [imported] = no and an empty [errors] column for every single row, even though the on-screen message says that column will contain row-level error information.

The failing request returned this (pulled from failing POST over the network):

There was an SQL error: ERROR: syntax error at or near ";"
LINE 15: LIMIT ?;

with a query ending:

AND "location_name"='Selwood (501Chelwood)'
...
ORDER BY _row_id
LIMIT ?;

My spreadsheet cell actually reads Selwood (?Chelwood). In the query it has become Selwood (501Chelwood). The question mark has been replaced by 501.

It looks as though values are substituted into the SQL first, and the resulting string is then scanned for ? placeholders. My literal question mark is taken for a placeholder and consumes the LIMIT argument, so the real LIMIT ? is left with nothing to bind.

I changed that one cell to Selwood (Chelwood) and changed nothing else in the file. 1 cell out of 15,895. The import then completed successfully. Same file, same settings.

One smaller thing. When the import fails this way, the errors file is produced with no row-level errors in it, which makes the failure very hard to diagnose from the interface alone. I only found the cause by looking at the network response. It might be worth surfacing the server error message in that file.

I am happy to supply the file, the errors CSV, or to re-run anything that would help you reproduce it.