Here's the thing. You said a "trilby is a fedora."
Is it also a hat? Yes. No one's arguing that.
As someone who is a classy g[ent][le]man who studies classy headwear, I am telling you, specifically, in science, no one calls trilbies fedoras. If you want to be "specific" like you said, then you shouldn't either. They're not the same thing.
If you're saying "hat family" you're referring to the categorical grouping of brimmed hats, which includes things from bowlers to homburgs to tophats.
So your reasoning for calling a trilby a fedora is because random people "call the big ones fedoras?" Let's get westerns and pork-pies in there, then, too.
Also, calling someone a human or an ape? It's not one or the other, that's not how fashion works. They're both. A trilby is a trilby and a member of the brimmed hat family. But that's not what you said. You said a trilby is a fedora, which is not true unless you're okay with calling all members of the brimmed hat family fedoras, which means you'd call bowlers, homburgs, and other hats fedoras, too. Which you said you don't.
It's okay to just admit you're wrong, you know?