GitHub - hellock/icrawler: A multi-thread crawler framework with many builtin image crawlers provided.
Documentation: http://icrawler.readthedocs.io/ Try it with pip install icrawler or conda install -c hellock icrawler. This package is a mini framework of web crawlers. With modularization design, it is easy to use and extend. It supports media data like images and videos very well, and can also be applied to texts and other type of files. Scrapy is heavy and powerful, while icrawler is tiny and fl
GitHub - serpapi/google-search-results-python: Google Search Results via SERP API pip Python Package
GitHub - s0md3v/Arjun: HTTP parameter discovery suite.
Arjun can find query parameters for URL endpoints. If you don't get what that means, it's okay, read along. Web applications use parameters (or queries) to accept user input, take the following example into consideration http://api.example.com/v1/userinfo?id=751634589 This URL seems to load user information for a specific user id, but what if there exists a parameter named admin which when set to
GitHub - kotartemiy/newscatcher: Programmatically collect normalized news from (almost) any website.
GitHub - phalt/django-api-domains: A pragmatic styleguide for Django API Projects
This styleguide combines domain-driven design principles and Django's apps pattern to provide a pragmatic guide for developing scalable API services with the Django web framework. This styleguide tries to tackle two big problems: Design philosophies and design patterns work in "ideal" situations, and most real life problems do not represent this ideal world. Therefore we need to develop a flexible
goop - Google検索結果をスクレイピング
Googleの検索結果を取得して分析に使いたいと考える人は大勢います。しかし機械的に収集しようとすると、GoogleからCAPTCHA入力が求められます。そのため自動化しづらく、手作業で収集している人も多いでしょう。 しかし裏道がありそうです。Facebookを経由するとそのトラップに引っかからないようです。その実証として作られたのがgoopです。 goopの使い方 goopで検索を行います。その際、Facebookのクッキーを適用するのがコツです。 from goop import goop page_1 = goop.search('open source', '<facebook cookie>') print(page_1) ちゃんと検索結果が返ってきます。 {0: { 'url': 'https://opensource.org/osd-annotated', 'text': '
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GitHub - tresni/pyHurricaneDNS: Python interface to manage Hurricane Electric DNS records ( http://dns.he.net )
GitHub - devanshbatham/Passivehunter: Subdomain discovery using the power of 'The Rapid7 Project Sonar datasets'
GitHub - StoicNZ/dns-probe: DNSDB Lookup and Resolver Python Script
GitHub - hangmansROP/securitytrailsapi: A wrapper around the Security Trails API
GitHub - dustyfresh/securitytrails-python
GitHub - D4-project/BGP-Ranking: BGP ranking is a free software to calculate the security ranking of Internet Service Provider (ASN)
GitHub - barnumbirr/ares: A python wrapper around https://cve.circl.lu.
Scripts/Security/cve-search.py at 7f22a727073ba1185e06b9ef42475f33279c645e · znb/Scripts
GitHub - bpb27/twitter_scraping: Grab all a user's tweets (and get past 3200 limit)
GitHub - jayswan/bgpview: CLI client for bgpview.io
GitHub - thisbejim/Pyrebase: A simple python wrapper for the Firebase API.
GitHub - Probely/API_Scripts: Probely API usage examples
GitHub - Te-k/pybinaryedge: Python 3 Wrapper for the BinaryEdge API https://www.binaryedge.io/
GitHub - fhightower/robtex-python: Simple python wrapper for the Robtex API (see https://www.robtex.com/api/).