While the relative level of formal authority vested in a project manager is greater in project-oriented (formerly projectized) organization structures than in matrix ones, the downside of this authority is that the project manager will spend much more time on people management administrative activities such as performance evaluations, hiring and supporting their professional development. While this is important work, it doesn't directly relate to the management of their projects and they might perceive it as a distraction.
In addition, in those organizations which are purely project-oriented (i.e. everything they do is project work with no functional or matrix structures to be found elsewhere within their walls), when projects end, if the team members who were contributing to them cannot be deployed to different projects then they may find themselves out of a job which is likely to stress their project managers even more at the very time when they are trying to line up other projects for themselves.
But there is a silver lining to this people management cloud.
Having these responsibilities will force the project manager to learn about the hopes, dreams and career aspirations of their direct reports. This should provide them with a greater ability to enable them to connect the team members' individual purposes to the success objectives of the project. They will also be better positioned to understand the competencies over which their staff wish to gain mastery which they can use to identify opportunities for personal development for these team members. Finally, even as project managers working in matrix structures will need to learn how to effective delegate, empowering their staff to work with autonomy is even more critical when there is a formal reporting relationship in place.
Project managers in project-oriented organizations might chafe at the additional responsibilities they have to shoulder, but these also give them more power to inspire their team members.