There are several places Outlook uses to store contacts. Your personal contacts are stored in folders either in an Outlook Data file or in your Exchange or Office 365 mailbox. Organizational contacts are stored in a Global Address List. You can add, edit, or delete contacts stored in your personal contacts folders. You can't add, edit, or delete contacts stored in the Global Address List. Contact your administrator for assistance with Global Address List contacts.
To delete a contact from one of your contacts folders:
On the navigation bar, click People.
Select the contact you want to delete and then click Delete on the Ribbon or right-click the contact and then click Delete.