It is a sad and known fact that having lots of ACLs in your SharePoint database has a negative effect on the performance of your site. What I did not know yet was that changes to the ACL will impact your incremental crawls.

Vikram Lakhotia has posted a description about this in his blog post: “SharePoint Incremental Crawl taking a long time after adding or removing a user”.