Explanation/Reference:
Explanation:
Ensure that files stored are larger, not smaller than 250MB.
You can have a separate compaction job that combines these files into larger ones.
Note: The file POSIX permissions and auditing in Data Lake Storage Gen1 comes with an overhead that
becomes apparent when working with numerous small files. As a best practice, you must batch your data
into larger files versus writing thousands or millions of small files to Data Lake Storage Gen1. Avoiding
small file sizes can have multiple benefits, such as:
Lowering the authentication checks across multiple files

Reduced open file connections

Faster copying/replication

Fewer files to process when updating Data Lake Storage Gen1 POSIX permissions

References:
https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-best-practices