Is not wanting your family to attend your wedding selfish?

I have always dreamt of a small private wedding with just the bride and groom on a beautiful beach abroad. My fiance likes this idea too and his family are not upset about it. We plan on having a large reception with family and friends as soon as we get back. I come from one of those families where big traditional weddings where family and family friends you've never even heard attend and that is the norm. Im not a shy person but Im very private about things concerning the heart and I know I would absolutely hate that. My dad is saying he wont get to talk me down the aisle and its left me feeling so bad. Are we being selfish and inconsiderate without realizing or something? What should I do?