How to Rank Any List by Picking Two at a Time (and How Many Picks It Takes)

By Kirill Zubovsky · · 5 min read

Quick: put your twelve favorite movies in order, from best to worst. If your brain just froze a little, you're normal. Ranking a whole list at once is hard. Choosing between two movies is easy.

That small difference is the idea behind pairwise ranking: you never rank the list. You answer a series of "this one or that one?" questions, and a computer (or a piece of paper) turns your answers into a complete order. This guide explains how that works, walks through a four-item example, and answers the question everybody asks: how many picks does it take? We measured it.

Why two at a time works

To give a list a numeric score, you need a scale in your head. Is the pizza a 7 or an 8? Is the movie night a 6.5? Those numbers shift with your mood. But "which would I rather have right now, pizza or ice cream?" has a quick answer almost every time.

Comparing two things also forces a real trade-off. You can't give everything a "pretty good," because you have to pick one. That makes pairwise choices useful for anything where the criteria are fuzzy: favorite foods, weekend plans, gifts, names, or which project to start first.

A worked example with four snacks

Say the four snacks are Pizza, Ice cream, Popcorn and Cookies. Sortpair uses a method called merge sort, which builds the order from small sorted groups. Here's what happens:

  1. Pair them up. Pizza vs Ice cream: you pick Pizza. Popcorn vs Cookies: you pick Cookies. That's two picks, and now you have two tiny sorted lists: Pizza, Ice cream and Cookies, Popcorn.
  2. Merge the lists. Compare the front item of each list: Pizza vs Cookies. You pick Cookies, so Cookies goes first. That's pick three.
  3. Keep going. The fronts are now Pizza vs Popcorn. You pick Pizza, so it goes second. That's pick four.
  4. Finish. The left list still has Ice cream and the right list still has Popcorn. Ice cream vs Popcorn: you pick Popcorn. That's pick five, and the last item falls into place.

Final order: 1. Cookies, 2. Pizza, 3. Popcorn, 4. Ice cream. Five picks for four items, and nothing was ever compared that didn't have to be. Notice you never compared Cookies with Ice cream. You didn't need to: Cookies beat Pizza, and Pizza beat Ice cream, so the order was already settled.

How many picks does it take?

The obvious way to rank a list is to compare every item with every other item. For n items that is n × (n − 1) ÷ 2 pairs, which grows quickly: 45 pairs for ten items, 190 for twenty. Nobody wants to do 190 taps.

Merge sort needs far fewer, because it uses what it already knows. To get real numbers for Sortpair, we ran its own sorting routine 20,000 times for each list size, answering every question at random. Here's what came out:

ItemsEvery pairTypical picksMost we saw
332.53
5106.68
72111.814
104520.525
1510538.545
2019056.969

Method: Sortpair's own merge routine, 20,000 simulated runs per size, random answers. "Typical" is the average. Real people aren't random, and lists that are already close to sorted need fewer picks, so your own count will vary a little.

A rule of thumb you can remember: at around ten items, expect about twice as many picks as items; shorter lists need less than that. Seven items is about a dozen picks; ten items is about twenty. That's a few minutes of tapping, which is why Sortpair works well for families and classrooms.

What if your picks contradict each other?

You might say you prefer A to B, B to C, and then C to A. In theory that's a loop with no "right" order. With merge sort it doesn't cause trouble, because the method only asks the questions it needs, so it rarely sees all three comparisons. It will always give you a complete list based on the picks you made.

A loop is still a clue. It usually means those items are close in your mind. If a decision matters, play a second round and see if the order changes. Items that swap places between rounds are your toss-ups.

What pairwise ranking can't tell you

It's worth being honest about the limits:

When you have clear numbers (price, time, distance), a simple table with scores may be better. When you only have feelings and fuzzy reasons, pairwise picks are hard to beat.

Tips for better rankings

Can you do it without an app?

Yes. Write each item on an index card. Take the first card and put it down. Take the second and compare it to the first, placing it above or below. For each new card, compare it with the middle card of your sorted row to decide which half it belongs in, then repeat inside that half. This is called binary insertion, and it needs about the same number of comparisons as merge sort. It's a nice activity for a classroom or a rainy day.

Ideas for things to rank

Frequently asked questions

Is pairwise ranking the same as an Elo rating?

No. Elo-style systems keep a running score across many matchups. Sortpair is simpler: it sorts your list once, using as few questions as it can, and gives you a fixed order at the end.

Does the order the pairs appear in matter?

Sortpair shuffles your list before it begins, so the first pair isn't always your first two items. Your own picks decide the result.

Is my list saved or shared?

Your list is saved in your own browser so it's there when you come back. Nothing is uploaded, and there's no account.

Can I rank more than one list?

Yes. Use Copy list to save your result anywhere you like, then press New list.

More from the Sortpair blog