The issue is probably not the stakes per se--it's that your neighbor seems to not agree with the boundary.
You could call the police on your neighbor, on the grounds they are trespassing. That may force matters to a head.
You could discuss matters with your neighbor--find out what their concern or issue is. Once you know it, you can try to resolve it. If there is a boundary dispute, you may need to go to court for an order fixing the boundary (a declaratory judgment; that is, a judgment declaring what the boundary is, or whose deed or survey is correct). If you and your neighbor can't work things out, only the courts can determine the proper boundary.
The Answer(s) provided above are for general information only. The attorney providing the answer was not serving as the attorney for the person submitting the question or in any attorney-client relationship with such person. Laws may vary from state to state, and sometimes change. Tiny variations in the facts, or a fact not set forth in a question, often can change a legal outcome or an attorney's conclusion. Although AttorneyPages.com has verified the attorney was admitted to practice law in at least one jurisdiction, he or she may not be authorized to practice law in the jurisdiction referred to in the question, nor is he or she necessarily experienced in the area of the law involved. Unlike the information in the Answer(s) above, upon which you should NOT rely, for personal advice you can rely upon we suggest you go to AttorneyPages.com
and retain an attorney to represent you.