If you look at the banner, at the very top, you will notice that 100 pixels isn’t a terribly large space. Anything less than 100 pixels would be too small to be seen, too hard to click on and too hard to manage.
However, you do get the opportunity to associate your icon with up to 80 characters of mouse-over text. This gives you an opportunity to describe your site with more than just an image.