In the "inverted pyramid" the most important information (which should come first) is represented by the base, which is the biggest part of the pyramid and holds up the rest of the pyramid. In a sense, it is the foundation, so you have to "get it right".
The analogy is "base = big = foundational = important"
Personally I think that's confusing, because you just as easily say the tip of the pyramid should represent the most important information, which should be conveyed concisely and without extraneous detail or background.
In that case the analogy would be, "tip = concise = main point = important"
That is, the lines in the heading got progressively shorter, making a visual inverted pyramid, with the most important information first.
Later, the "inverted pyramid" term described the structure of the entire article with the most important parts first, but the metaphor does seem backward.
https://books.google.com/books?id=rNaEw8DwatwC&pg=PA154&dq=%...
I think of a pyramid from the ground up, so a dense base followed by a thinner top.
A inverted pyramid would be thin first then dense and large.
When reading though, you go from top to bottom, so if you're more visual instead of time based, you may see it the other way around.
The triangle is upside down:
If anything a regular pyramid makes more sense to me: you want the smallest/narrowest useful description at the top and then you gradually expand on it as you go down, providing more (wider) context and detail for the key information.
Edit: Of course, it's a widely used term and good to understand in that context; the Wikipedia link is useful.
This is often the case with geometric metaphors. They catch on easily, but they rarely make a lot of sense on closer scrutiny.