tekeousA to Lemmy Shitpost@lemmy.world · 1 year agorobots.txt is a suggestionimagemessage-square25fedilinkarrow-up1625arrow-down13
arrow-up1622arrow-down1imagerobots.txt is a suggestiontekeousA to Lemmy Shitpost@lemmy.world · 1 year agomessage-square25fedilink
minus-squareunalivejoy@lemm.eelinkfedilinkEnglisharrow-up5·1 year agoPeople really should be providing a sitemap.xml file
People really should be providing a sitemap.xml file