I do a copy into MapD from an s3 bucket. However the copy finishes, having only done about a third of the files in the s3 bucket. The result from the command line mentions 0 rejected, it is as if it has ignored a large amount of files in the s3 bucket.
I have double checked the s3 permissions and all is fine.
The s3 bucket has around 3000 files (compressed CSV). The data set is around 1.2 billion rows, but this is just my initial test before I use a much larger sized dataset.
Is there is a number-of-files limit in MapD when doing a COPY (from s3)? It may be a coincidence but I would guess it is failing around the 1000 file mark