Robots.txt File Overrides Robotnik

[Nicholas-Westby]

Seems like the file system version of a robots.txt is favored over a Robotnik robots.txt. The workaround is to ensure there is not robots.txt on the file system, but would be good to find a way to force the Robotnik configuration to be used.

[Nicholas-Westby]

Might require a handler like this (configured in the web.config):

<handlers accessPolicy="Read, Write, Script, Execute">
  <remove name="Robots"/>
  <add name="Robots" verb="*" path="/robots.txt" type="SomeAssembly.SomeHandler, SomeAssembly" preCondition="managedHandler" />
</handlers>

Hopefully there is a way using code to accomplish this, however. Otherwise, a web.config transformation will be necessary on install.