Language (translation) and gerography come into play. On the one hand babka is sometimes translated (mistranslated?) into English as egg bread. But Anglos overuse the word bread --- things such as what they call nut bread,date bread, etc. would be called keks (fruit cake) or ciasto orzechowe, and toast bread goes by the name of pieczywo tostowe. So much for language.
I have heard that in old SE Poland (now Ukrainian-occupied territory) at Easter ham was eaten with babka so it played the role of a bread without being called that. In general Poles don't feel the need to label babka as a bread or a cake -- it's just babka, pure and simple.