2012-12-23

My problem with pagination

I recently got into a discussion about a comment I made on a blog post two years ago. Granted; my comment might have come out a bit harsh (for which I apologize) but there was a point to be made. In this blog post I'll try to explain what I was (and am) talking about and what my issues were (and are) with the proposed solution to the problem at hand: pagination of large chunks of data / resultsets.

Pagination of data

I'll start of with the biggest issue I have with the pagination of data: It's a solution for a problem that shouldn't exist in the first place. If you need to present large amounts of data to a user so that you end up needing 2,300 pages of results requiring the user to go to page 543 then that is the actual problem. Let's assume that we have a relatively small "page size"; meaning that we present 10 results per page. If you have 2,300 pages that means 23,000(!) results the user needs to wade through to find what he/she needs. How is the user expected to know that what he/she is looking for is at page 543 in the first place? And that's only when we assume a "best case scenario" of 10 results per page, not 20, 25, 50 or even a 100 which is also not very uncommon.

What the user needs is not a "slider" to page results, what the user needs is a way to filter, sort and even search the results to narrow it down to a few results; maybe a page or 2, 3. I don't have numbers and haven't tried to look them up but I'm going out on a limb here and bet that 98% of the users googling for stuff never even reach page 2 of the results for their query. The other 2% that actually do dig deeper will maybe page through a few of the pages but I'm pretty sure nobody in their right mind ever went to page 543 of the search results for a google query. What people will do is try various permutations of their query in a quest to find their desired page they are looking for.

Let me be clear; I am not saying paging is a bad idea altogether and neither am I saying that huge resultsets or datasets are or even that when the user insists on sifting through thousands of pages of results that he/she should be prohibited to do so. I just think it is very hard to come up with a scenario where this would be the case; where it would benefit the user to be confronted with this large amount of data to find the needle in a haystack and where the "best solution" would be the user to page through result after result.

The proposed slider

Unfortunately the screencast in the original blog post is offline at the moment I'm writing this so the actual demonstration is lost. However, the demo of a blogpost that started the discussion is still accessible. The latter, however, shows something a little different: this slider to "page through data" seems like a nice "widget" to browse, for example, a handful or maybe even up to a 100 photos in an "album" or "collection" of some sort. I do have some issues with that slider as well though.

First: it requires me to click the number in-between the left and right angles (which users are accustomed to, although I would prefer a bit more explicit "previous / next" sort of hint) to show (or: unhide) the slider. Second: It depends on the willingness of a user to "explore" what can be done instead of being intuitive. I do understand that, from a designers point of view, always displaying the "slider" would possibly not be as aesthetic as the proposed slider but it would be a lot more intuitive when the page number would be displayed on a "line" immediately instead of after the user clicking the number. Also; when a user would click one of the angles he/she would notice the page number move on the line (e.g. the slider would be moving) showing them there could be more to it than just a circle with a number in it; the secret of the slider would unveil itself. It also would cost a click less. Showing (or rather: unhiding) the slider on mouseover for the entire "paging area" would also help but that would mean the angles moving outward which would require the user to hit a moving target when they desire to click the angles. This, however, could easily be fixed by simply fixing the angles in the final position in the first place.

Slider as a paging concept

Another problem I have with a slider as a paging concept is that while it might work okay for a few dozen of pages, it will become a problem sooner-or-later when the resultsets do get bigger (which, as I explained earlier on, you'd want to avoid in the first place). Let's go back to the 2,300 pages of results and applying a slider to this. Let's say we have a really wide slider of 1,000 pixels. That would work out to be a "resolution" of 2.3 pages for each pixel a user can scroll. Each time you move the slider with your mouse for a single pixel, which is very hard to do for most users in the first place, you skip a page or more. The user would see the pagenumber go from 1 to 3 to 6 to 8 to ... for each pixel moved. The internals of the slider would count 1, 3.3, 5.6, 7.9, ...and would require us to round to the nearest integer resulting in pages skipping in a 2, 3, 2, ... kind of way. Not very intuitive for the user. Let's assume we find a creative way to make this more consistent; then we would still be faced with the fact that the user can't easily go to, for example, the desired page 7. That would involve the user "sliding" to page 6 or 8 and next use one of the "angle-buttons". This effect even gets worse for smaller sliders (which, for design purposes, might be limited to a few hundred of pixels high or wide) and matters will become even worse when the number of resultpages go up (which, again, is what started the discussion in the first place).

A slider does have it's function and may even be applied as a paging mechanism in some scenarios. In the scenarios where we have many pages, however, it would worsen instead of improve the UX. However, and this the 'takeaway' of this blogpost, I do think that when we get at the point where we need to page data (or results, patients, eventlog items, whathaveyou) the best option to help your user find their desired data (patient, eventlogitem, ...) is to provide a way to filter and/or search this data. To narrow it down to a handfull of records that can be grasped and processed by the users mind instead of flooding it. A filter, a means of searching and possibly even sorting data is, in my humble opinion, in most cases a better option than paging.

Alternatives? Solutions?

As a developer (I'm no UX expert in any way, nor am I a designer) I too have been confronted with this problem. And, to be totally honest, in many cases I did just "stick a pager" at the bottom of the page. That's because it's the easiest way out. I can also by now tell you that I don't pretend to have a good alternative or solution to when you actually do want users to be able to page through enormous amounts of data. Some alternatives have been cropping up the past several years like the "never ending page" as I like to call it, or, as it is usually called the "endless scrolling page". This works fine for data like Facebook posts, Twitter timelines, 9Gag posts etc. where users usually aren't looking for something very specific, just browsing (or wading) through the data as they go along. It might even work in more "serious" scenarios where, for example, a history of financial data is presented. Each problem needs to be looked at and then fitted with the best solution for the problem; for financial data this might, for example, be a line graph showing the data much more concise and allowing users to find patterns like plummeting sales. I'm not saying you need to reinvent the wheel for each problem but applying the correct wheel is an art in and of itself for many developers.

Once, a long time ago, I did try to address the pagination issue myself and I'm willing to show what I came up with for you to laugh at and criticize. Please note that I do not propose this as a solution and that what you're about to witness is not something a good "UX expert" would come up with. All I want to do is demonstrate that, despite this blogpost, I also do come up with some bad less than brilliant ideas (and sometimes even more horribly executed).


Unfortunately the pagination is in dutch ("vorige" meaning "previous", "volgende" meaning "next" and ... well, you'll get the idea...). I too was faced with having either a big-ass paging area showing 318 or more pages (or: links to the pages) and wanting to give the user a way to quickly jump to a specific page. I would have decorated the pagenumber with some sleek curved arrow and a little explanation in a few words to tell the user they could click the pagenumber but the further I executed my idea the more I was convinced this was not going to end up being a good UX. I ended up keeping this "feature" as a "hidden feature" only used by those that were "initiated". The rest of the users will probably never know. And I don't mind; currently they have been given the gift of (extended) searching/filtering/sorting and thus diminishing the need for pagination in the first place.

So, in conclusion, I will reiterate a part of my earlier comment: "Sliders are great for “volume controls” or stuff that doesn’t require exact input.". And I hope most of you agree. You don't? Leave a comment below and let's discuss!

Update

I have been thinking about this topic a little more and the more I do, the more I realize that the analogue equivalent is a great way to demonstrate my objections to pagination. Imagine if you will a coworker dropping a 2,300 page report (or worse) on your desk asking you to "have a look at that weird discrepancy on the sales of product X around August". I'm already having trouble imagining a 2,300-page report but bear with me. If you're lucky the report is sorted by date and contains sales of exactly one year; so to find what you're looking for you split that stack at about 8/12th or 2/3rd and you should be looking at data at-or-around August... if you're lucky. Then it's a matter of sifting through about 191 pages to find the numbers you're looking for. If product X happens to be a barbecue then you won't sell many of them in January and sell more in summer. Here's where it starts to break down; you won't be able to adequately guess where to start in the report so you'll need to find a way to quickly flip through the pages. Maybe you can. Maybe you're that good a bookworm. Then, if you're lucky, you might find what you're looking for in a few minutes. If the report, however, also happens to be a report of stores around the country each reporting their sales then you're in trouble: you're going to have to go through the entire stack, picking pages containing the sales numbers you're looking for and even setting those pages apart. Things start to get messy with several stacks of paper everywhere. And it's not hard to imagine even worse case scenarios where you'll finally end up hiring extra workers just to process these huge piles of reports because these reports tend to be generated every once in a while. This is what happens in 2012; data is churned out in massive amounts around the world every second. Companies, governments, individuals, karts tracks and weather stations and everyone and everything produces data which is stored, accessed and published in all kinds of ways. You can't compare this to anything back in the day, when paper ruled. Being able to manipulate (e.g. search, sort, filter, group etc.) these amounts of data is why we have these amounts of data in the first place. Imagine the FBI, CIA or whatever looking for a terrorist that happened to have posted on Facebook. Imagine Facebook honoring a request for data on IP and other logged stuff requiring someone (or rather: a team of hundreds of people) having to go through massive amounts of paper, probably stored in several warehouses, to find those two records that got produced that one day when terrorist X happened to log on and post something. Unimaginable if you ask me. It should require some intern logging in to System Y and punching in some numbers and producing a report.

So, I think the analogue equivalent makes my point even more clear: paging, pagination in any kind of form, essentially is some leftover from the good old days that somehow stuck with us into the digital age. Maybe we find it reassuring to keep this analogue equivalent around, maybe it never even came to our minds that paging is a silly idea. And yet we have thought of WHERE clauses in SQL, come up with clickable table headers to sort, have great search-engines etc. etc.  And still we keep building pagination and, what started the discussion, finding "creative ways" to get rid of "pagenumbers" replacing them with sliders. Maybe it's time for a change. Let's get rid of that barbaric pagination and address the need for it instead. And if we're going to paginate data, let's try to keep it down to a handful of pages, not 2,300 requiring someone to find the needle in this pile'o'dead-wood at page 543. And, again, please do note that I'm not referring to pages of blogposts for example where you're "inviting" the user to browse the posts. I'm referring to pages-and-pages of eventlogitems, sales, financial data, customers, products etc. where the user is not browsing but looking for specific items in these pages.

Update January 25th (a little over a month later)

Stefano Gargiulo replied December 24th:
and January 4th I inquiried on his progess:
however, unfortunately, I haven't heard anything from him since. A shame, I would've loved a constructive discussion. Ah, well...