I think I am already at that point for silver, at least in bulk. Having bought rolls of eagles for between $200 and $300, I can't seem to get excited about the prospect of buying the same rolls at $600+. I will still buy individual numismatic silver at this price.
As for gold, I feel similar, but I am sure there is a number I wouldn't be comfortable paying. Is that number $2,000, $2,500? Certainly $5,000 would be absurd.