Should you go to prison if you've been found looking at child pornography?

Writer John Grisham says there are men in prison who "...would never touch a child" but "...pushed the wrong buttons, went too far and got into child porn."