Skip to content

Instantly share code, notes, and snippets.

View paulmwatson's full-sized avatar

Paul Watson paulmwatson

117 followers · 146 following
View GitHub Profile
@paulmwatson
paulmwatson / string_url.js
Last active June 7, 2021 12:37
Manipulating the URL via strings
let url = 'https://host.test/path/here.filetype?query=string&more=params#anchor';
const querystringStart = url.indexOf('?');
const anchorStart = url.indexOf('#');
const querystring = url.slice(querystringStart + 1, anchorStart);
let querystringParams = querystring.split('&');
let newQuerystringParams = [];
querystringParams.forEach((queryparam, i) => {
let param = queryparam.split('=');
if (param[0] !== 'more') {
newQuerystringParams.push(`${param[0]}=${param[1]}-changed`)
@paulmwatson
paulmwatson / cdxt_commoncrawl_example_domain.sh
Created May 25, 2021 10:46
Use CDXT Toolkit to search CommonCrawl for certain domains/URLs
cdxt --cc --limit 100 --filter '=status:200' iter 'github.com/*'
@paulmwatson
paulmwatson / idp_copy_event_and_actions.py
Created April 20, 2021 09:01
Copies an IDP event and its actions.
from idp_data.idp_data.models import Event, EventAction
id = '103'
original_event = Event.objects.get(pk=id)
new_event = Event.objects.get(pk=id)
new_event.pk = None
new_event.id = None
new_event._state.adding = True
new_event.save()
@paulmwatson
paulmwatson / python_assignment_expressions.py
Created April 9, 2021 08:32
Python assignment expressions (if/conditional assignments)
d = {'a': True, 'b': False}
# Without assignment expressions
if d['a']:
a = d['a']
print(a)
# => True
if d['b']:
b = d['b']
@paulmwatson
paulmwatson / date.parse.js
Created March 4, 2021 07:29
// Firefox
Date.parse('2021-03-03 06:00')
//=> 1614744000000
// Chrome
Date.parse('2021-03-03 06:00')
//=> 1614751200000
// Safari
Date.parse('2021-03-03 06:00')
@paulmwatson
paulmwatson / main.rb_main.rbs
Created January 27, 2021 20:20
Ruby 3 RBS dynamic typing
#main.rb
class Super
def initialize(val)
@val = val
end
def val?
@val
end
end
@paulmwatson
paulmwatson / twitter_birdwatch_note_ranking.py
Created January 26, 2021 07:54
Python code to replicate the Twitter Birdwatch Note Ranking algorithm
#Credit: https://twitter.github.io/birdwatch/about/ranking-notes/
import pandas as pd
notes = pd.read_csv("notes-00000.tsv", sep="\t")
ratings = pd.read_csv("ratings-00000.tsv", sep="\t")
ratingsWithNotes = notes.set_index("noteId").join(
ratings.set_index("noteId"), lsuffix="\_note", rsuffix="\_rating", how="inner"
)
ratingsWithNotes["numRatings"] = 1
@paulmwatson
paulmwatson / github_actions_docker_nektos_act.yaml
Created January 22, 2021 10:10
Use nektos/act to test Github Actions with Docker locally
#.github/workflow/test.yaml
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Test
run: |
@paulmwatson
paulmwatson / dockerfile_multi_stage_build
Created January 21, 2021 14:13
Docker multi-stage build with large files
#Dockerfile
#model_artifacts image
FROM python:3.7.7-slim-buster AS model_artifacts
WORKDIR /artifacts
ADD https://bucketname.s3-eu-west-1.amazonaws.com/really_big_model_1.h5 .
ADD https://bucketname.s3-eu-west-1.amazonaws.com/really_big_model_2.h5 .
#app image
FROM python:3.7.7-slim-buster AS app
...
@paulmwatson
paulmwatson / square_comparison.js
Created January 18, 2021 09:21
Comparing squaring.
//example1.js
const func1 = (foo, bar)=> {
const num1 = Math.pow(foo, 2)
const num2 = Math.pow(bar, 2)
}
//example2.js
const func1 = (foo, bar)=> {
const num1 = foo **2
const num2 = bar **2