[privoxy-devel] TLS Pool could add HTTPS to Privoxy

[Rick van Rein]

Hello,

On your TODO list I noticed item #16 on support of HTTPS. May I propose
99% of the solution?

I'm working on a TLS Pool daemon, which concentrates all the sensitive
handling of credentials in a process separate from the application
logic. This process is initiated by passing sockets over a
POSIX-standard mechanism to the daemon, and have the daemon handle
handshaking, encryption and decryption.

Privoxy could be a plaintext filter in between an upstream and
downstream HTTPS connection. Downstream (to the client/browser) there
is a need to add a certificate on-the-fly, signed under a root cert that
the client would need to accept. This on-the-fly certification has been
specified for the TLS Pool and will be implemented soon.

Would this be a good solution for Privoxy? One counter-argument could
be that an extra software component is required; that component however,
is of general use and has been designed with callback APIs that provide
a lot of control over the TLS connection to the users. That includes
authentication through X.509, PGP, SRP and Kerberos, all with
credentials protected by a PKCS #11 API. There is even a framework
coming up for centralised configuration of TLS policies. Probably a bit
more than you would probably put in if you added TLS only for proxying :)

So I wonder, is this direction of interest to Privoxy?

https://github.com/arpa2/tlspool
https://tlspool.arpa2.net


Best wishes,

Rick van Rein
ARPA2.net

------------------------------------------------------------------------------

[Fabian Keil]

Post by Rick van Rein
On your TODO list I noticed item #16 on support of HTTPS. May I propose
99% of the solution?
I'm working on a TLS Pool daemon, which concentrates all the sensitive
handling of credentials in a process separate from the application
logic. This process is initiated by passing sockets over a
POSIX-standard mechanism to the daemon, and have the daemon handle
handshaking, encryption and decryption.

Interesting.

Post by Rick van Rein
Privoxy could be a plaintext filter in between an upstream and
downstream HTTPS connection. Downstream (to the client/browser) there
is a need to add a certificate on-the-fly, signed under a root cert that
the client would need to accept. This on-the-fly certification has been
specified for the TLS Pool and will be implemented soon.
Would this be a good solution for Privoxy?

Without additional information that's hard to tell.

Are you proposing to provide patches or that other people write them?

I briefly looked at the tlspool repository and it's not obvious to me
that it's free software. Where and what is the license?

Fabian

[Rick van Rein]

Hi Fabian,

Post by Fabian Keil
Interesting.

That's the most important thing I was hoping to hear :)

Post by Fabian Keil

Post by Rick van Rein
Would this be a good solution for Privoxy?

Without additional information that's hard to tell.
Are you proposing to provide patches or that other people write them?

We have an open source development fund for TLS Pool and could probably
fund its integration into Privoxy, if it is considered useful. That
could take the shape of an external code donation, or ask someone in the
Privoxy project to do it for us. We're open to any proposal that is
reasonable. As I said, the most important thing to me is that it is not
considered "too foreign" to be acceptable. We can talk details, but the
main idea is that TLS is concentrated into a daemon of general use,
rather than having yet another application that requires its own TLS
implementation/configuration -- that is what the TLS Pool in general
wants to get away from.

As for our motivation for cooperation :- our project would be
demonstrable with Privoxy, making the authentication/encryption
mechanisms we added to the TLS Pool available to at least web users; we
cannot currently demonstrate that without an HTTPS proxy. Privacy
filtering is an added value that fits well with this kind of
demonstration, I think.

Post by Fabian Keil
I briefly looked at the tlspool repository and it's not obvious to me
that it's free software. Where and what is the license?

Excellent question. It *definately is* open source software, but you
are right, I should pick one of BSD, GPL, LGPL. These are in fact the
only three options that we'd consider for the TLS Pool. What you're
seeing is that the software isn't in a 1.0 yet (although its
functionality is tested). I will see to this.

Cheers,
-Rick

------------------------------------------------------------------------------

[Rick van Rein]

Hi Fabian & list,

Post by Fabian Keil
I briefly looked at the tlspool repository and it's not obvious to me
that it's free software. Where and what is the license?

Thanks for pointing out this oversight. I added the needed files, you
can find them from
https://github.com/arpa2/tlspool/blob/master/COPYRIGHT.MD

Summary:
* daemon code: GPLv3
* user space: 2-clause BSD
* documentation: CC BY-SA 4.0

Cheers,
-Rick

------------------------------------------------------------------------------

[Rick van Rein]

Hi all,

FWIW, the TLS Pool now has a demonstration HTTPS_PROXY that shows how easy it is to setup TLS in an intervening proxy. Browsers still seem to be missing flags or certificate attributes, but that's trivial stuff; the fundamental interaction works like a charm.

https://github.com/arpa2/tlspool/blob/master/tool/https_proxy.py

This is Python; the C parallel is similiar. If you guys/girls are interested to integrate this form of TLS into Privoxy, then talk to me!

Cheers,
-Rick

[Ian Silvester]

Thanks Rick,

I suspect that this will be attractive to our project. When you say the
C parallel do you mean that the code is developed in parallel in two
different languages? Whilst clearly there exists HTTP(S) as a
language-agnostic 'API' interface between the two softwares, I
personally have a preference for compiled C over Python.

Cheers,

Ian

Post by Rick van Rein
Hi all,
FWIW, the TLS Pool now has a demonstration HTTPS_PROXY that shows how easy it is to setup TLS in an intervening proxy. Browsers still seem to be missing flags or certificate attributes, but that's trivial stuff; the fundamental interaction works like a charm.
https://github.com/arpa2/tlspool/blob/master/tool/https_proxy.py
This is Python; the C parallel is similiar. If you guys/girls are interested to integrate this form of TLS into Privoxy, then talk to me!
Cheers,
-Rick
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers

[Rick van Rein]

Hello Ian,

2 February 2016 at 23:02
Thanks Rick,
I suspect that this will be attractive to our project.

:-)

When you say the C parallel do you mean that the code is developed in
parallel in two different languages? Whilst clearly there exists
HTTP(S) as a language-agnostic 'API' interface between the two
softwares, I personally have a preference for compiled C over Python.

Same here.

The TLS Pool is a daemon written in C, communicating with TLS-using
applications like Privoxy over a UNIX domain socket (or a variation on
non-POSIX a.k.a. Windows). Privoxy could directly use the sockets or
application-support wrappers in C and Python, the last of which is used
in this demo.

Simple example programs written in C, not an HTTPS proxy but a simple
chat client and server over TLS, are testcli and testsrv,
https://github.com/arpa2/tlspool/blob/master/tool/testcli.c
https://github.com/arpa2/tlspool/blob/master/tool/testsrv.c

The simplicity of the TLS Pool comes from separation of logic:
* applications like Privoxy only know about streams and (to be) proven
identities
* protocol settings are setup in databases, but not by the application
* optional user-interfacing code can guide local identity selection and
PIN entry

Cheers,
-Rick

P.S. This is work in progress, scheduled for release on July 1st.
Specifically, the validation framework is something I still need to work
on; but that is all isolated from applications like Privoxy.

[Ian Silvester]

That's great Rick, thanks for the detailed info.

We will discuss internally (well, likely on this list) and, if there is
general agreement, move forward with interoperability testing. If that
goes well there is the potential to document TLS Pool as our recommended
method for implementing HTTPS filtering support in Privoxy.

Cheers,

Ian

Post by Rick van Rein
Hello Ian,

2 February 2016 at 23:02
Thanks Rick,
I suspect that this will be attractive to our project.

:-)

When you say the C parallel do you mean that the code is developed in
parallel in two different languages? Whilst clearly there exists
HTTP(S) as a language-agnostic 'API' interface between the two
softwares, I personally have a preference for compiled C over Python.

Same here.
The TLS Pool is a daemon written in C, communicating with TLS-using
applications like Privoxy over a UNIX domain socket (or a variation on
non-POSIX a.k.a. Windows). Privoxy could directly use the sockets or
application-support wrappers in C and Python, the last of which is used
in this demo.
Simple example programs written in C, not an HTTPS proxy but a simple
chat client and server over TLS, are testcli and testsrv,
https://github.com/arpa2/tlspool/blob/master/tool/testcli.c
https://github.com/arpa2/tlspool/blob/master/tool/testsrv.c
* applications like Privoxy only know about streams and (to be) proven
identities
* protocol settings are setup in databases, but not by the application
* optional user-interfacing code can guide local identity selection and
PIN entry
Cheers,
-Rick
P.S. This is work in progress, scheduled for release on July 1st.
Specifically, the validation framework is something I still need to work
on; but that is all isolated from applications like Privoxy.

[Fabian Keil]

Post by Rick van Rein
FWIW, the TLS Pool now has a demonstration HTTPS_PROXY that
shows how easy it is to setup TLS in an intervening proxy.
Browsers still seem to be missing flags or certificate attributes,
but that's trivial stuff; the fundamental interaction works like a charm.
https://github.com/arpa2/tlspool/blob/master/tool/https_proxy.py

Interesting.

Post by Rick van Rein
This is Python; the C parallel is similiar. If you guys/girls are
interested to integrate this form of TLS into Privoxy, then talk to me!

FYI, I'm thinking about trying to raise some funding to work on TODO
list item #16 ("Filter SSL encrypted content as well") within the next
months.

I'm not sure if using TLS pool is the best option, but I'll certainly
look at it.

Fabian

[Rick van Rein]

Hello Fabian & Privoxy Developers,

A while back, we talked about adding TLS to Privoxy through my TLS Pool
daemon. If this is still of interest to Privoxy then it may be good to
know that funding for such things is currently available;I learnt that
NLnet is currently quite interested in TLS-related project inquiries,
and you might be able to collect some independent funding for such an
enhancement.

https://nlnet.nl/news/2016/20161201-call-en.html

I don't know if you had already seen this, but I wanted to make sure
you'd seen it. The deadline for a project request would be December
1st; it is a low-threshold funding process aimed specifically at open
source projects.

FWIW, the TLS Pool has a facility for "on the fly" signing with a
signing cert that could be signed under a root cert installed by the
remote party. Packaging is currently in progress.


Cheers,
-Rick

Post by Fabian Keil

Post by Rick van Rein
FWIW, the TLS Pool now has a demonstration HTTPS_PROXY that
shows how easy it is to setup TLS in an intervening proxy.
Browsers still seem to be missing flags or certificate attributes,
but that's trivial stuff; the fundamental interaction works like a charm.
https://github.com/arpa2/tlspool/blob/master/tool/https_proxy.py

Interesting.

Post by Rick van Rein
This is Python; the C parallel is similiar. If you guys/girls are
interested to integrate this form of TLS into Privoxy, then talk to me!

FYI, I'm thinking about trying to raise some funding to work on TODO
list item #16 ("Filter SSL encrypted content as well") within the next
months.
I'm not sure if using TLS pool is the best option, but I'll certainly
look at it.
Fabian

------------------------------------------------------------------------------

[Fabian Keil]

Post by Rick van Rein
Hello Fabian & Privoxy Developers,
A while back, we talked about adding TLS to Privoxy through my TLS Pool
daemon. If this is still of interest to Privoxy then it may be good to
know that funding for such things is currently available;I learnt that
NLnet is currently quite interested in TLS-related project inquiries,
and you might be able to collect some independent funding for such an
enhancement.
https://nlnet.nl/news/2016/20161201-call-en.html

TLS support is still of interest and I already proposed a
"TLS/SSL support for Privoxy" project to NLnet in May.

NLnet confirmed the receipt but since then I haven't heard
back from them.

IIRC the deadline was June or July when I made the proposal.
I don't know if NLnet silently rejected the proposal or simply
extended the deadline to December.

Post by Rick van Rein
FWIW, the TLS Pool has a facility for "on the fly" signing with a
signing cert that could be signed under a root cert installed by the
remote party. Packaging is currently in progress.

That certainly sounds useful.

Fabian