Hi John,
The Layer.RemoveShapes/Lines methods are used to move elements between layers, they do not remove elements from the diagram itself. These functions are called internally when the diagram is cleared.
When talking about the Build routine, can I assume this is the insertion and shapes and lines into a diagram, and not the Visual Studio build?
You should always Suspend and Resume the diagram before inserting elements, otherwise each change is rendered and this can impact performance. You should suspend the UndoList unless you actually plan to use undo/redo, this can consume memory, usually performance issues are related to.net having to allocate more memory.
Finally, do you load data from a database, attach to any events etc, this could be causing long running items. A code profiler (such as ones available from Red Gate) can identify these issues.