humungus - miniwebproxy

i am gravely disappointed

there has been too much violence

overview - files - changes

111:ecf89679307a on 2019-08-21 23:32:45 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: tip

just hostname here, minus port, should make things easier to work with

110:3a518d8c5476 on 2019-08-13 17:31:08 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

rework filtering to provide some more features. try to document some of it even.

109:13d24fec8950 on 2019-08-13 16:23:35 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

simply close code a bit

108:42bdd9ba3918 on 2019-05-06 00:08:11 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

update permitted tags

107:eb4595c0222d on 2019-04-21 02:37:25 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

go.mod build

106:4fce3676492f on 2019-04-21 02:31:34 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

filter to chew on toots as well

105:7238d11a3008 on 2019-03-29 10:38:49 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

the main tag may be just the tag we're searching for

104:aa7305dc13b4 on 2019-03-29 08:49:44 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow id attribute, so #urls work

103:97e428504e7b on 2019-03-18 01:40:28 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

repair accidental commit to filter.lua

102:716f74ab310e on 2019-03-17 00:02:58 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

connect to port 80 is not tls. also add back a bit of logging.

101:729c973f7982 on 2019-03-16 21:13:19 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.9 for changeset c9662fb3d18c

100:c9662fb3d18c on 2019-03-07 05:54:44 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.9

attempt to deal with websockets. maybe it works.

99:c628a41412f8 on 2019-03-07 04:12:29 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

forgot, easier to just peek than read/unread

98:cbf01ee074fd on 2019-03-07 02:45:07 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

use a semi custom transport for roundtrip

97:474533847da0 on 2019-03-07 02:34:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

simply code a bit. notably, http.transport does a better job handling
idle connections. this seems faster and more reliable.

96:7da945c3e9a8 on 2019-03-06 23:44:35 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

keepalive means we need better timeout handling

95:0a0c57454caf on 2019-03-06 07:50:25 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

keep the connection open (for https).
reduce useless logging.

94:8eb249211290 on 2019-02-17 23:46:21 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

slip the command line filter into the main exec

93:cbd4f6489b34 on 2019-02-16 08:58:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

replace some silly writestring(sprintf) calls with fprintf

92:e019d060bcbd on 2019-02-14 21:41:58 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

include some more information in the error message sent to browser

91:0645f631c15b on 2019-02-14 05:52:12 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.8 for changeset ce93d20d49b5

90:ce93d20d49b5 on 2019-02-14 05:51:27 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.8

update readme to latest style

89:f3ebcf339db4 on 2019-02-09 00:39:32 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

a tweak here and there to the filter rules

88:a9dffb52f2bb on 2019-02-06 23:07:50 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

copyright

87:d9bf148b8b53 on 2019-02-06 19:11:31 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.7 for changeset 185ab726a7c3

86:185ab726a7c3 on 2019-02-06 19:09:14 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.7

silly to keep release.sh here

85:1ca3f54c087d on 2019-02-06 05:04:12 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

sometimes content is in .content

84:4e4586688580 on 2019-02-06 04:56:06 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

a bit more precision about head vs body

83:617bcd628e81 on 2019-02-06 04:41:58 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow link tags such as found in head

82:c37f354d51a0 on 2019-02-06 04:29:01 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

reformat some code

81:4889092776c9 on 2019-02-06 04:22:01 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

chunked transfer encoding

80:ec4aa2a53c7b on 2019-02-06 04:13:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

in case we're not gzipping, use a bufio

79:c1ca1af16325 on 2019-02-06 03:18:36 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

possibly faster to escape and write in one pass

78:d27154536eb3 on 2019-02-06 01:18:34 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow tables to have rowspan and colspan

77:e8c947e81753 on 2019-02-06 01:13:38 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

include links to embedded youtube videos instead of hiding

76:441879680eb7 on 2019-02-05 18:41:19 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

the html we write will always be utf-8. prevent encoding mismatches.

75:60a881a55615 on 2019-01-31 05:30:53 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

a function to save unfiltered url for debugging troublesome sites

74:a7a3d5f01a1d on 2019-01-30 00:16:12 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

still need to send 200 when not intercepting

73:19d90b1a0aae on 2019-01-29 21:36:31 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

move outbound connection up earlier so we can report connect errors

72:f28b9ea6f7f6 on 2019-01-26 08:13:25 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

tweak the locking to be a bit more efficient

71:54e059527372 on 2019-01-26 08:06:21 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

rename some variables to make things easier to follow

70:47ea1a33bf28 on 2019-01-26 02:39:35 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

start using context and deadlines. incomplete.

69:9b564e604721 on 2019-01-25 21:18:01 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

openbsd defaults to 2048 bit key, but make it explicit

68:533ff0a1b979 on 2019-01-25 21:16:58 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

clean up the cert cache expiry code a bit

67:0b7b20e95709 on 2019-01-25 09:50:26 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

stop using responsewriter after hijacking

66:39a32f662a75 on 2019-01-25 08:19:39 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

move a function

65:86af52ab5f26 on 2019-01-25 07:06:58 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.6 for changeset dba7d89250ad

64:dba7d89250ad on 2019-01-25 07:06:17 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.6

oops, long standing bug in luainterface. forgot to pop return.
broke filtering after shared interpreters change.

63:164837d976ce on 2019-01-24 11:22:03 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.5 for changeset 43cd9b74804d

62:43cd9b74804d on 2019-01-24 11:07:26 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.5

one lua interpreter per request should be enough

61:337150fb25e5 on 2019-01-24 11:00:54 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

rework the filtering pass to avoid all the temp buffers.
should be bit leaner and faster now.

60:82a534fd2cef on 2019-01-22 12:43:09 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

fixes and tweaks to filtering,
mostly to get busted blogspot pages working

59:8e8de1b52985 on 2019-01-22 05:00:19 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

govendor does a better job of vendoring

58:e8b8be4268dd on 2019-01-20 22:59:44 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

some sites use brotli, which isn't decoded. for now, hack around this
by forcing gzip accept-encoding instead of adding dependencies.

57:cdf3b25fe8ba on 2019-01-20 22:50:09 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

add a few more rules to filter.lua

56:b37b3156fb8f on 2019-01-20 22:49:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

tweak rules to be simpler

55:b584c78fa4cd on 2019-01-18 02:06:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

add a command line filter for testing and experimentation

54:abf64af5b136 on 2019-01-16 11:52:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.4 for changeset 360e7f70b8b7

53:360e7f70b8b7 on 2019-01-16 11:52:07 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.4

clean up release dir a little

52:fb53e7298eb7 on 2019-01-16 11:50:08 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.3 for changeset 14024fd1ccf8

51:14024fd1ccf8 on 2019-01-16 11:49:47 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.3

always name output miniwebproxy

50:020731045977 on 2019-01-16 11:45:08 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.2 for changeset fb5ca99cad8d

49:fb5ca99cad8d on 2019-01-16 11:45:00 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.2

rename resign.sh to just sign.sh

48:7e035cab1f46 on 2019-01-16 11:43:56 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.1 for changeset ea3aa57467fb

47:ea3aa57467fb on 2019-01-16 11:42:50 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.1

release machinery

46:4eb8ad0180e5 on 2019-01-16 11:21:16 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

Added tag v0.9.0 for changeset 5d050db048ae

45:5d050db048ae on 2019-01-16 11:09:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Tagged: v0.9.0

improve makefile

44:5e23f47c109c on 2019-01-16 07:37:12 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

update some notes

43:1222668343bc on 2019-01-16 07:33:40 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

not used here, but support more types in the luainterface reflector

42:1133d3a09316 on 2019-01-15 13:45:57 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

do not pass bodybytes to lua. it's huge and inefficient.

41:ccf7b75479bd on 2019-01-15 12:11:29 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow the proxy to use a proxy, such as tor.

40:9878222b5971 on 2019-01-11 10:45:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

resort imports

39:23083bb802e9 on 2019-01-11 10:21:15 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

go fmt

38:7b58cce48648 on 2019-01-11 10:17:31 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

filtering on github and amazon is usually bad

37:c3b7a3eacc8e on 2019-01-11 10:17:02 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

remove annoying zipr div

36:d56a40b6a232 on 2019-01-11 10:16:41 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

proxy is supposed to do connection: close

35:b0e80c5c6872 on 2019-01-01 22:41:38 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

sup

34:1a322662576d on 2019-01-01 03:32:42 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

make sure outbound dial has a port

33:af463b194af6 on 2019-01-01 03:20:53 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

more bugs fixed by simply reusing request in http case

32:6f156798a685 on 2018-12-31 19:59:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

h4-h6 are ok tags

31:909ab243d0e2 on 2018-12-31 06:48:02 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

use io.WriteString instead of []byte casts

30:437c340fa0f0 on 2018-12-31 06:19:33 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

copy all the headers for http proxy. should just write it back.

29:1a56300de514 on 2018-12-29 18:53:17 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

stupid times has articles that inconsistently put img in figure
but sometimes leave it out.

28:6934120c56dc on 2018-12-28 22:57:12 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

add a simple signing script

27:a7593cc2cef9 on 2018-12-27 21:55:07 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

some hosts do not like seeing a port number in host. strip it.

26:fcb2d7177cf6 on 2018-12-26 04:50:30 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

need to copy cookies for plain http.
probably need to do more, but this moves stuff along.

25:b8dc83d2e08d on 2018-12-24 22:55:02 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

seems we should care about errors creating certs

24:457709242321 on 2018-12-24 20:02:02 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow to force reddit to old.reddit.com

23:33295d890e66 on 2018-12-24 18:49:33 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

can use init function to sort arrays

22:f9cdde536621 on 2018-12-24 18:25:49 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

refine permitted tags a big. more table stuff

21:2cea426d5fd7 on 2018-12-06 05:11:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

link to noscript parse bug issue

20:90b8971637bc on 2018-12-06 05:06:00 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

s is an allowed tag

19:5e8f62f4c390 on 2018-12-06 05:05:24 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

remove obsolete filtering code. it lives in lua now.

18:d193d249ae19 on 2018-11-21 23:30:50 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Parent: 16:d91b9f1883de

m

17:4e99fa132268 on 2018-11-21 23:29:56 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Parent: 9:1732887fe591

belated status update

16:d91b9f1883de on 2018-01-13 16:03:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

repair some image urls that hide in data attrs

15:1164da4ee688 on 2018-01-13 10:25:38 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

allow skipping http too

14:fd7f751bf2f6 on 2018-01-12 17:02:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

don't print inside of script tag

13:59cdfd81a6aa on 2018-01-12 13:47:54 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

sometimes the gist shit isn't in an iframe, but just a script

12:df5863a71ea6 on 2018-01-12 13:47:02 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

a few more selectors to find an article

11:05442c988748 on 2018-01-12 13:36:34 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Parent: 9:1732887fe591

m

10:91e663507a32 on 2018-01-12 13:35:50 +0000 UTC by Ted Unangst <tedu@tedunangst.com>
Parent: 4:0e7d2596575e

move shouldfilter logic into lua

9:1732887fe591 on 2017-12-12 06:32:37 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

quick fix for multiple articles; don't rewrite yet

8:b8ef96dd7c2f on 2017-12-12 06:29:05 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

small fix to grab more header info

7:522c943acca4 on 2017-12-12 05:51:18 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

some sites like using picture. try to make it an img

6:1c2d98f8189a on 2017-12-12 04:08:49 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

much more aggressive about article filtering

5:8a9ba430d277 on 2017-12-12 04:08:28 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

quick fix to get filtering for plain http as well

4:0e7d2596575e on 2017-12-12 01:31:55 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

make lua filters a bit more useful

3:a7820b23fba8 on 2017-12-10 02:24:51 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

start moving the filtering logic into a lua script

2:5e78fe97de58 on 2017-10-31 19:59:45 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

a case study in regridding homepages, with weird json special sauce

1:001533479baf on 2017-10-25 20:11:32 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

parse nearly all html responses.
allows filtering by indicators other than url.

0:df53184ac9cd on 2017-10-24 03:13:13 +0000 UTC by Ted Unangst <tedu@tedunangst.com>

miniwebproxy