Web Crawling & Web Scraping

A large scale web crawler to take an overview about security of Moroccan sites

Security Analyzer A large scale web crawler (considered also as vulnerability scanner tool) to take an overview about security of Moroccan sites Current architecture / Workflow: Implemented defence mechanisms scanners: X-Frame-Options HTTP Strict-Transport-Security X-Content-Type-Options Secure cookies HttpOnly cookies CSRF Tokens Content security policy Iframe sandboxing Implemented vulnerabilities scanners: X-XSS-Protection Mixed-content inclusions Vulnerable remote JavaScript inclusion SSL-stripping Vulnerable Form Outdated Server Software – Outdated CMS Sensitive files HTTP Parameter Pollution Information leakage Insecure SSL implementation Usage Check the examples in the […]

Read more
Web Crawling & Web Scraping

Python Crawler written Python 3

pycrawler Python Crawler written Python 3. (Supports major Python releases Python3.6, Python3.7 and Python 3.8) Installation and Use Setup VirtualEnv which python3 this will output the path of your python3 #now setup a python3 virtualenv mkvirtualenv crawl3 -p $(which python3) workon crawler python main.py -d5 http://gotchacode.com // -d5 means crawl to the depth of 5. Results: And the output is: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:00<00:00, 29200.11it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 22563.50it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 21375.28it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 22227.37it/s] CRAWLER STARTED: https://vinitkumar.me, will […]

Read more
Asyncio

An async python framework with the backend runtime written in rust

Robyn Robyn is an async Python backend server with a runtime written in Rust, btw. Python server running on top of of Rust Async RunTime. Installation You can simply use Pip for installation. pip install robyn Usage from robyn import Robyn app = Robyn() @app.get(“/”) async def h(): return “Hello, world!” app.start() Contributor Guidelines Feel free to open an issue for any clarification or for any suggestions. If you’re feeling curious. You can take a look at a more detailed […]

Read more
Advanced, Classification, NLP, Programming, Project, Python, Supervised, Technique, Text, Use Cases

Amazon Product review Sentiment Analysis using BERT

This article was published as a part of the Data Science Blogathon Introduction Natural Language processing, a sub-field of machine learning has gained immense popularity in the last 5 years in both research and industrial applications due to the advancement in the field of deep learning and improvement in the computational power of hardware systems. It is a technique for computers to understand how human languages work involving the usage of computational linguistics and the computer science domain. In recent years, […]

Read more
Beginner, Machine Learning, NLP, Unstructured Data, Unsupervised

Topic Modeling and Latent Dirichlet Allocation(LDA) using Gensim and Sklearn : Part 1

This article was published as a part of the Data Science Blogathon Introduction Let’s say you have a client who has a publishing house. Your client comes to you with two tasks: one he wants to categorize all the books or the research papers he receives weekly on a common theme or a topic and the other task is to encapsulate large documents into smaller bite-sized texts. Is there any technique and tool available that can do both of these two […]

Read more
Machine Learning

CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds

CloudAAE This is an tensorflow implementation of “CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds” Files log: directory to store log files during training. losses: loss functions for training. models: a python file defining model structure. object_model_tfrecord: full object models for data synthesizing and visualization purpose. tf_ops: tensorflow implementation of sampling operations (credit: Haoqiang Fan, Charles R. Qi). trained_network: a trained network. utils: utility files for defining model structure. ycb_video_data_tfRecords: synthetic training data and real […]

Read more
Machine Learning

Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition

Patch-NetVLAD This repository contains code for the CVPR2021 paper “Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition” The article can be found on arXiv and the official proceedings. Installation We recommend using conda (or better: mamba) to install all dependencies. If you have not yet installed conda/mamba, please download and install mambaforge. conda create -n patchnetvlad python=3.8 numpy pytorch-gpu torchvision natsort tqdm opencv pillow scikit-learn faiss matplotlib-base -c conda-forge conda activate patchnetvlad We provide several pre-trained models and configuration […]

Read more
Face recognition

A standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions. Evaluation metric Given a single monocular image, the challenge consists of reconstructing a 3D face. Since the predicted meshes occur in different local coordinate systems, the reconstructed 3D mesh […]

Read more
PyTorch

Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking

ArTIST Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking (CVPR 2021) Pytorch implementation of the ArTIST motion model. In this repo, there are Training script for the Moving Agent network Training script for the ArTIST motion model Demo script for Inferring the likelihood of current observations (detections) Demo script for Inpainting the missing observation/detections Demo 1: Likelihood estimation of observation Run: python3 demo_scoring.py This will generate the output in the temp/ar/log_p directory, look like this: This demo gets as […]

Read more
Translation

Break-It-Fix-It: Learning to Repair Programs from Unlabeled Data

BIFI Break-It-Fix-It: Learning to Repair Programs from Unlabeled Data This repo provides the source code & data of our paper: Break-It-Fix-It: Unsupervised Learning for Program Repair (ICML 2021). @InProceedings{yasunaga2021break, author = {Michihiro Yasunaga and Percy Liang}, title = {Break-It-Fix-It: Unsupervised Learning for Program Repair}, year = {2021}, booktitle = {International Conference on Machine Learning (ICML)}, } Problem: Repair Task Our approach: BIFI 0. Dependencies Specifically, run the following commands to create a conda environment (assuming CUDA10.1): conda create -n BIFI […]

Read more
1 … 588 589 590 591 592 … 921