I suppose the reason one can say that the universe is not fractal in nature, even though we observe variations on a theme all around us, is that it is not a perfect fractal. When you examine things closely, they are irregular, they don't follow a specific mathematical formula in terms of their similarities and there are no laws of physics that provide a framework for a fractal universe.
This came to mind when I was working on a bit of graphics in PhotoShop. It was a low resolution graphic that I was trying to clean up. On screen, at its original size, it looks nice and smooth, but when you expand it and look at it pixel by pixel, it's quite chaotic. The nice smooth lines become jagged and irregular, even discolored. The image becomes almost unrecognizable.
This is one of those things where I'm pretty sure there's a connection to be made. I just haven't found it. Little help?