.Google.com confines robots.txt assistance to four industries, clarifying its own viewpoint on in need of support ordinances.Google just assists 4 certain robots.txt areas.Unsupported regulations in robots.txt will certainly be neglected.Look at bookkeeping your robots.txt data taking into account this update.