{"id":455,"date":"2014-01-21T15:30:05","date_gmt":"2014-01-21T14:30:05","guid":{"rendered":"http:\/\/www.phigtech.com\/?p=455"},"modified":"2014-01-27T15:30:20","modified_gmt":"2014-01-27T14:30:20","slug":"squidguard-et-les-blacklists","status":"publish","type":"post","link":"https:\/\/www.phigtech.com\/?p=455","title":{"rendered":"SquidGuard et les Blacklists"},"content":{"rendered":"

J’ai r\u00e9cemment du impl\u00e9menter des listes noires dans squid via squidGuard.<\/p>\n

Le probl\u00e8me, c’est que certaines listes sont tellement grosses qu’il est presque impossible de les compiler sans bloquer le serveur. De plus, les performance sont d\u00e9grad\u00e9es sur ces listes.<\/p>\n

Vient alors la question: sont-elles \u00e0 jour ?<\/p>\n

pour les nouveaux sites, plut\u00f4t. Par contre, il traine dedans moultes sites qui n’existent plus.<\/p>\n

d’o\u00f9 le petit script suivant qui permet de cr\u00e9er un fichier .new de la blacklist en param\u00e8tre (chemin) .<\/p>\n

#!\/bin\/bash<\/span>
\n
\nfunction<\/span> one_line (<\/span>)<\/span>{<\/span>
\nif<\/span> host -s<\/span> -4<\/span> -W<\/span> 1<\/span> -t<\/span> any $1<\/span> >\/<\/span>dev\/<\/span>null 2<\/span>>&<\/span>1<\/span> ; then<\/span>
\necho<\/span> "$1"<\/span> >><\/span>$2<\/span>.new
\nok<\/span>="ok"<\/span>
\nelse<\/span>
\necho<\/span> "-$1"<\/span>>><\/span>$2<\/span>.diff
\nok<\/span>="nok"<\/span>
\nfi<\/span>
\necho<\/span> "$1 $ok<\/span>"<\/span>
\n}<\/span>
\n
\ncount<\/span>=0<\/span>
\nNB_PROC<\/span>=8<\/span>
\n# nb de process en parallele<\/span>
\nfor<\/span> line in<\/span> $(<\/span>cat<\/span> $1<\/span>)<\/span>
\ndo<\/span>
\none_line $line<\/span> $1<\/span> &<\/span>
\nlet<\/span> count+=1<\/span>
\n[<\/span>[<\/span> $(<\/span>(<\/span>count%<\/span>NB_PROC)<\/span>)<\/span> -eq<\/span> 0<\/span> ]<\/span>]<\/span> &&<\/span> wait<\/span>
\n#si modulo 0 tous process pris!<\/span>
\ndone<\/span><\/div><\/div>\n

libre \u00e0 vous de le mette ensuite en production en ajoutant au script :<\/p>\n

squidGuard -C<\/span> $1<\/span><\/div><\/div>\n

si vous utilisez le nouveau fichier ( .new) que vous prendrez soin de renommer ou<\/p>\n

squidGuard -u<\/span><\/div><\/div>\n

si vous utilisez le .diff g\u00e9n\u00e9r\u00e9 \ud83d\ude42<\/p>\n

vous noterez la parall\u00e9lisation des tests: la commande host \u00e9tant plut\u00f4t lente, il est int\u00e9ressant de traiter plusieurs lignes en simultan\u00e9.<\/p>\n

 <\/p>\n

 <\/p>\n

<\/div>