First, three quick checks

Legs. Count the legs. Insects have six and three body sections. Eight legs means a spider, mite or tick; many legs means a centipede or millipede; fourteen on a small gray oval is a pill bug, which is a crustacean.
Size. Measure it, or photograph it beside a coin. Memory makes every bug bigger.
Waist, wings and antennae. A pinched waist and elbowed antennae say ant or wasp. A thick waist and straight antennae on a winged swarmer say termite.

Then, where did you find it?

Kitchen and pantry

Small beetles in flour or cereal, tiny flies over fruit or the sink.

Weevils and flour beetles: The snout tells you which group you have.
Fungus gnat vs fruit fly: Where it came from answers it faster than how it looks.
Water bug vs cockroach: Water bug is a nickname, not a species.

Bathroom, basement and drains

Damp brings the moisture lovers: fast, flat, many-legged or fuzzy-winged.

Silverfish: A silverfish is a wingless, flat, teardrop-shaped insect covered in silvery scales, with long thread-like antennae and three tail-like appendages at the rear.
Drain fly: A drain fly is a tiny, fuzzy, moth-like gnat that sits still on the wall near a sink or a floor drain and makes short hopping flights when disturbed.
House centipede: The fast, many-legged thing in a bathroom is almost always a house centipede.
Camel cricket: A camel cricket is the humpbacked, wingless, long-legged cricket that jumps out of a basement corner.
Earwig: An earwig is the flat, dark reddish brown insect with a pair of claw-like pincers at the rear.

Bedroom, wardrobe and carpets

Small, oval and slow usually means a beetle, not what you fear.

Bugs that look like bed bugs: Size sorts most of it.
Carpet beetle: Carpet beetles are small oval beetles, 1/16 to 1/8 inch long, that Kentucky extension describes as black or mottled with white, brown, yellow and orange.

Windowsills and sunny walls, especially in autumn

Insects looking for somewhere to spend the winter.

Asian lady beetle vs ladybug: Both are lady beetles.
Boxelder bug: A boxelder bug is a black true bug with thin red lines, about half an inch long, that gathers in numbers on the sunny side of a wall in autumn.
Stink bug: The brown marmorated stink bug is a shield-shaped brown bug about 17 mm, or two thirds of an inch, long and almost as wide as it is long.

Swarming, flying or nesting

Winged ants or termites in spring, striped fliers around the eaves in summer.

Termite vs flying ant: Three features separate them.
Wasp vs hornet vs yellow jacket: Size, color and the nest separate them.

Questions people ask

How do I find out what bug this is?

Note where you found it, how big it is, and how many legs and wings it has, then photograph it from above with something for scale. Where it turned up narrows the list fast: pantry insects, damp-loving insects and overwintering insects are three different casts. A photo then settles most of what is left.

Should I be worried?

Usually not. Most insects found indoors wandered in and cannot live there. What matters is whether it damages wood, fabric or stored food, or stings, and that depends entirely on what it is, which is why identifying it comes first.