Richard Sherman is big...so I am not sure how you discount him.
Jason Verrett, the guy who has played 24 of 48 games in his three year career?
Let's just go through it
AJ Bouye - 6'
Delvin Breaux - 6'1
Malcolm Butler - 5'11
Vontae Davis - 5'11
Stephon Gilmore - 6'1
Joe Haden - 5'11
Chris Harris - 5'10
Janoris Jenkins - 5'10
Trumaine Johnson - 6'2
Johnathan Joseph - 5'11
Josh Norman - 6'
Marcus Peters - 6'
Patrick Peterson - 6'1
Jalen Ramsey - 6'1
Darrelle Revis - 5'11
Xavier Rhodes - 6'1
Richard Sherman - 6'3
Darius Slay - 6'
Aqib Talib - 6'1
Desmond Trufant - 6'
Jason Verrett - 5'10
So, in that list, 3 guys come in at 5'10. Of those guys, Verrett ran a sub 4.4, and the rest ran mid 4.4's.
So, of the (roughly) top 21 CBs in the NFL, 3 are shorter than 5'11, and one of those three ran under 4.4, and all three under 4.5.
8 of the top 21 are under 6'. Less than 50%.
Can and likely are two different things. Over 50% of the top 21 CBs in the NFL are 6' tall or over, and 86% are 5'11 or over. I am guessing that size is not overrated after all.