When people think about stateful web applications, I believe that most of the concerns usually focus on the scalability of that web application. Stateful web applications are more difficult to scale out, as you need to ensure that all requests from the same session are handled by the same application instance, or you need to setup some mechanism so that instance A can pick up and resume the session that was started by instance B, creating a potential bottle neck.
However, with the new breed of web applications out there that allow a session to be sending multiple, possibly concurrent requests (via practices like AJAX), a new crop of concerns arise. Most (if not all) web containers take care of managing concurrency concerns between sessions. But, most do nothing to manage concurrency concerns within the same session.
Java concurrency master Brian Goetz provides the details at http://www.ibm.com/developerworks/java/library/j-jtp09238.html