The ones who walk away from Omelas aren't unable to accept that one must suffer greatly for others to be happy -- they're miffed that the nanny state government of Omelas doesn't capitalize on the concept further by getting a thousand suffering children. Imagine how much more utopia that'd create!