Attachments added via the New Comment displays twice in the Attachments related list
You are uploading attachments on the Email to Case Premium New Comment page, and notice that the attachment displays twice in the Attachments related list. You are also utilizing the Salesforce Lightning Experience UI when this occurs.
When attachments are added via the New Comment page and sent out via an email, the attachment will be added both to the email and at the Case level. When this is performed, the Attachments (and Google Docs & Attachments) related list in Lightning reads this as two separate attachments.
As Salesforce is planning to deprecate attachments within the Lightning Experience, the suggested solution is to adjust your Email to Case Premium to utilize Salesforce Files. Some more information on this switch and the configuration options available can be found here.