This version of OneNote doesn’t include all of the formatting features available in other version of OneNote.
You can change page color, but to change other colors, do the following:
To change notebook color, use OneNote for Windows
In OneNote 2016 for Windows, click File and then open the Settings > Properties for your notebook. Choose your notebook color from the dropdown.
In OneNote for Windows 10, right-click the notebook in the notebook list and choose Notebook Color.
To change section color, use OneNote for Mac or OneNote for Windows
Right-click the section in the section list, and choose Section Color.