"Simplicity" means different things, depending on the audience.

As a casual developer, I find WordPress moderately simple -- I know how to quickly get a job done using WordPress, and I'm sure that my speed at shipping code would be much worse were I to attempt the same things in Drupal or Joomla. While I know how to develop themes, organizing their structure and files are still complex (but not hard) affairs.

As a user, WordPress *can* be simple -- if you're a savvy-enough user and aren't distracted by the full breadth of out-of-the-box functionality in the Admin interface, it's simple enough to start publishing immediately. But some users may be overwhelmed by the Admin interface. And if WordPress isn't already installed on their host, they may find it challenging to install (much longer than 5 minutes). And it's easy enough for any user to install stale plugins or poorly-designed themes. And when it comes to Widget management, I personally go to my happy place. So from a user perspective, it can easily appear complicated, depending on the flavor of said user.

So I think it's less of a debate about whether WordPress is simple or not (it's *simpler* than what you might have been forced to do in the past when web publishing), there is room for alternatives like Tumblr, Ghost, etc. which remove barriers for certain user populations.