With fossil bone, you will be able to see the different canals and webbed structure of the bone, sure signs that the object was of biological origin.
As my good friend Mike Boyd notes, without going into the distinction between dermal bone and endochondral bone — which relates to how they form - or ossify — it's worth noting that bones such as the one illustrated here will usually have a layer of smooth (or periosteal) bone on the outer surface and spongy (or trabecular) bone inside.
The distinction can be seen in this photograph. The partial weathering away of the smooth external bone has resulted in the exposure of the spongy bone interiors. Geographic context is important, so knowing where it was found is very helpful for an ID.
Knowing the geologic context of your find can help you to figure out if you've perhaps found a terrestrial or marine fossil. Did you find any other fossils nearby? Can you see pieces of fossil shells or remnants of fossil leaves? Things get tricky with erratics. That's when something has deposited a rock or fossil far from the place it originated. We see this with glaciers. The ice can act like a plough, lifting up and pushing a rock to a new location, then melting away to leave something out of context.