These are accounting terms, not project management terms. They are good ones to know.
"Capitalized costs" are accrued over a long period of time (usually the useful life of the work product), and slowly hit the balance sheet of the company. Basically, it takes longer for the cost of the project to start reducing the profit (or increasing the loss) of the company.
"Expenses" hit the balance sheet of the company immediately, as soon as the money is spent or incurred. Project costs will immediately reduce company profit or increase loss.
Whether project costs can be expensed or capitalized is an accounting decision. Talk to your finance or accounting people to understand why they are capitalizing or expensing costs on the project. Sometimes certain project costs will be capitalized and others are expensed. It all depends on the nature of the project and the accounting practices of the company. Saving Changes...