Deep dive into multi-page recursive crawling mechanisms, page count budgeting, and domain scoping limits.