There may be other ways but this is how I do it. To delete, highligt the custom view in the Orientation dialog and press the delete key. To rename, double click custom view, press New View button, enter new name then delete old custom view.
For anyone having issues. Make sure you access the more options arrows,
& not the new view icon in the view orientation box. The more options in the view
orientation box enables removing past views etc. This should also be able to be accessed
by the space bar.