No way to avoid but there is an option to display duplicates, here:
And you can vote for the feature to avoid adding them