Discussion:
[privoxy-devel] Release date for Privoxy 3.0.24
Fabian Keil
2016-01-12 15:21:24 UTC
Permalink
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.

Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.

If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).

I'll be at FOSDEM at the end of the month and at AKtiVCongrEZ
the weekend thereafter, and thus wouldn't want to do a release
to close to theses dates.

Any objections?

Fabian
Ian Silvester
2016-01-12 16:16:16 UTC
Permalink
No objections from me.
Post by Fabian Keil
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.
Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.
If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).
I'll be at FOSDEM at the end of the month and at AKtiVCongrEZ
the weekend thereafter, and thus wouldn't want to do a release
to close to theses dates.
Any objections?
Fabian
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Lee
2016-01-12 21:08:55 UTC
Permalink
Post by Fabian Keil
Any objections?
nope
Post by Fabian Keil
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.
Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.
If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).
I'll be at FOSDEM at the end of the month and at AKtiVCongrEZ
the weekend thereafter, and thus wouldn't want to do a release
to close to theses dates.
Any objections?
Fabian
Fabian Keil
2016-01-15 14:06:09 UTC
Permalink
Post by Fabian Keil
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.
Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.
If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).
FYI, the CVS servers has been locked for a while now and
the SF website is even more broken than usual:

| We're sorry -- the Sourceforge site is currently in Disaster
| Recovery mode, and currently requires the use of javascript to
| function. Please check back later.
http://sourceforge.net/

Fabian
Ian Silvester
2016-01-15 16:33:37 UTC
Permalink
Sheesh - we really need to migrate elsewhere eh :-s
Post by Fabian Keil
Post by Fabian Keil
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.
Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.
If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).
FYI, the CVS servers has been locked for a while now and
| We're sorry -- the Sourceforge site is currently in Disaster
| Recovery mode, and currently requires the use of javascript to
| function. Please check back later.
http://sourceforge.net/
Fabian
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Ian Silvester
2016-01-16 15:15:06 UTC
Permalink
Fabian,

IIRC, last time around Sourceforge's SVN tagging was broken - on the offchance that is still the case, please let us know when you consider the source is 'frozen' for this release.

Cheers,

Ian
Post by Ian Silvester
Sheesh - we really need to migrate elsewhere eh :-s
Post by Fabian Keil
Post by Fabian Keil
I'd like to release Privoxy 3.0.24 in the near future after
flushing a few more commits. Hopefully this will be the last
release before migrating away from SourceForge.
Currently the weekend in two weeks (2016-01-23 - 24) looks good
to me, provided I get all the commits in before the end of the
upcoming weekend and no complex regressions are found.
If that doesn't work out, I'd like to postpone the release
until the second weekend in February (2016-01-13 - 14).
FYI, the CVS servers has been locked for a while now and
| We're sorry -- the Sourceforge site is currently in Disaster
| Recovery mode, and currently requires the use of javascript to
| function. Please check back later.
http://sourceforge.net/
Fabian
------------------------------------------------------------------------------
Post by Fabian Keil
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
Post by Fabian Keil
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
------------------------------------------------------------------------
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
------------------------------------------------------------------------
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
Fabian Keil
2016-01-17 15:02:25 UTC
Permalink
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on the
offchance that is still the case, please let us know when you consider
the source is 'frozen' for this release.
Will do.

I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.

Fabian
Lee
2016-01-17 18:38:15 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on the
offchance that is still the case, please let us know when you consider
the source is 'frozen' for this release.
Will do.
I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball? If no,
would you also do that?

Thanks,
Lee
Fabian Keil
2016-01-17 19:06:57 UTC
Permalink
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on the
offchance that is still the case, please let us know when you consider
the source is 'frozen' for this release.
Will do.
I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball? If no,
would you also do that?
I consider it a separate step, but intend to do it on Friday as well.

Fabian
Lee
2016-01-17 19:13:34 UTC
Permalink
Post by Fabian Keil
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on the
offchance that is still the case, please let us know when you consider
the source is 'frozen' for this release.
Will do.
I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball? If no,
would you also do that?
I consider it a separate step, but intend to do it on Friday as well.
Thank you. I know it shouldn't make any difference, but I still have
the feeling that building from a source tarball is more reliable than
getting the source code via cvs

Lee
Ian Silvester
2016-01-17 19:30:45 UTC
Permalink
Hi all,

I've discovered this morning that my build environment is quite broken. Yes, it fully serves me right - I should know that upgrading OS X would trample anywhere it likes in the filesystem without warning. I'll work on fixing it of course, but the OS X binary release might be delayed.

Separately, let me know each time you upload a new release file to Sourceforge's download folder hierarchy so that I can update my mirror.

Cheers,

Ian
Post by Ian Silvester
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on
the
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
offchance that is still the case, please let us know when you
consider
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
the source is 'frozen' for this release.
Will do.
I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball? If no,
would you also do that?
I consider it a separate step, but intend to do it on Friday as well.
Fabian
------------------------------------------------------------------------
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
------------------------------------------------------------------------
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
Ian Silvester
2016-01-18 22:01:25 UTC
Permalink
Hi all,

I've fixed my build environment so an OS X release is back on schedule.
One item of note is that Apple has replaced gcc with LLVM/clang, which
throws the following warning on compilation:

errlog.c:429:18: warning: absolute value function 'abs' given an
argument of type 'long' but has parameter of type 'int' which may
cause truncation of value [-Wabsolute-value]
this_thread = abs(this_thread % 1000);
^
errlog.c:429:18: note: use function 'labs' instead
this_thread = abs(this_thread % 1000);
^~~
labs

Clearly the sizes of long and int are platform dependent, but I don't
know if the code change it is suggesting is viable.

Of perhaps academic interest is the growth in size of the binary -
3.0.23 source compiled to x86_64 with gcc (4?) resulted in a 385064k
binary, whilst 3.0.24 compiled with LLVM 7 results in a 446120k binary!


Cheers,

Ian
Post by Ian Silvester
Hi all,
I've discovered this morning that my build environment is quite
broken. Yes, it fully serves me right - I should know that upgrading
OS X would trample anywhere it likes in the filesystem without
warning. I'll work on fixing it of course, but the OS X binary release
might be delayed.
Separately, let me know each time you upload a new release file to
Sourceforge's download folder hierarchy so that I can update my mirror.
Cheers,
Ian
IIRC, last time around Sourceforge's SVN tagging was
broken - on the offchance that is still the case,
please let us know when you consider the source is
'frozen' for this release.
Will do. I just pushed the ChangeLog and documentation
updates for 3.0.24 and intend to tag the tree on Friday
without any changes except for typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball?
If no, would you also do that?
I consider it a separate step, but intend to do it on Friday as well.
Fabian
------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
------------------------------------------------------------------------
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
-- My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Fabian Keil
2016-01-19 17:30:24 UTC
Permalink
Post by Ian Silvester
I've fixed my build environment so an OS X release is back on schedule.
Great.
Post by Ian Silvester
One item of note is that Apple has replaced gcc with LLVM/clang, which
errlog.c:429:18: warning: absolute value function 'abs' given an
argument of type 'long' but has parameter of type 'int' which may
cause truncation of value [-Wabsolute-value]
this_thread = abs(this_thread % 1000);
^
errlog.c:429:18: note: use function 'labs' instead
this_thread = abs(this_thread % 1000);
^~~
labs
Clearly the sizes of long and int are platform dependent, but I don't
know if the code change it is suggesting is viable.
Given that the code is in an "#ifdef __MACH__" block I have
no strong opinions on how to deal with the warning and if
it should happen before the release.

While using labs() seems viable, it's not obvious to me that the
whole line is (still) useful. If it isn't, removing it may be an
even better idea.

Could you try that and check how the thread ids in the log messages change?

Fabian
Ian Silvester
2016-01-19 19:34:57 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
I've fixed my build environment so an OS X release is back on schedule.
Great.
I spoke slightly too soon.

Whilst I have a working binary it seems that the version I have of
Apple's own package building tool no longer runs on my system. Apple's
monolithic approach to dev tool distribution means I cannot upgrade it
without risking the loss of down-level target SDKs. I am therefore
evaluating a free alternative and have high hopes of transitioning to it.

At this rate I'll be a few days behind the curve, but I don't see a
problem with that.
Post by Fabian Keil
Post by Ian Silvester
One item of note is that Apple has replaced gcc with LLVM/clang, which
errlog.c:429:18: warning: absolute value function 'abs' given an
argument of type 'long' but has parameter of type 'int' which may
cause truncation of value [-Wabsolute-value]
this_thread = abs(this_thread % 1000);
^
errlog.c:429:18: note: use function 'labs' instead
this_thread = abs(this_thread % 1000);
^~~
labs
Clearly the sizes of long and int are platform dependent, but I don't
know if the code change it is suggesting is viable.
Given that the code is in an "#ifdef __MACH__" block
I didn't look into the source file, assuming it was a different level of
strictness from LLVM; clearly I should have done!
Post by Fabian Keil
I have
no strong opinions on how to deal with the warning and if
it should happen before the release.
While using labs() seems viable, it's not obvious to me that the
whole line is (still) useful. If it isn't, removing it may be an
even better idea.
Could you try that and check how the thread ids in the log messages change?
Okay, will do.

Ian
Post by Fabian Keil
Fabian
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Ian Silvester
2016-01-20 16:08:36 UTC
Permalink
I have tested removal of the get_thread_id MACH IFDEF. It is indeed
strictly unnecessary since even without it the threads are still
uniquely identified. I propose however that it is kept and modified as
per the compiler warning, since the raw IDs use four extra bytes which
could better be used by the remainder of the log line, plus the comment
in the code is strictly true.

Let me know if anyone disagrees.

Here are log snippets before and after the IFDEF removal:


2016-01-19 23:11:43.935 000003c8 Info: Privoxy version 3.0.24
2016-01-19 23:11:43.936 000003c8 Info: Program name: /usr/local/sbin/privoxy
2016-01-19 23:11:43.944 000003c8 Info: Loading filter file:
/usr/local/etc/privoxy/default.filter
2016-01-19 23:11:43.956 000003c8 Info: Loading filter file:
/usr/local/etc/privoxy/user.filter
2016-01-19 23:11:43.957 000003c8 Info: Loading actions file:
/usr/local/etc/privoxy/match-all.action
2016-01-19 23:11:43.958 000003c8 Info: Loading actions file:
/usr/local/etc/privoxy/default.action
2016-01-19 23:11:43.971 000003c8 Info: Loading actions file:
/usr/local/etc/privoxy/user.action
2016-01-19 23:11:43.983 000003c8 Info: Loading actions file:
/usr/local/etc/privoxy/regression-tests.action
2016-01-19 23:11:43.997 000003c8 Error: block action without reason
found. This may become a fatal error in future versions.
2016-01-19 23:11:44.000 000003c8 Info: Listening on port 8118 on IP
address 127.0.0.1
2016-01-19 23:11:46.782 00000128 Request: api.simperium.com:443/
2016-01-19 23:11:47.027 00000368 Request: api.simperium.com:443/
2016-01-19 23:11:47.792 000001c0 Request: notify3.dropbox.com:443/
2016-01-19 23:12:05.013 00000018 Request: client-cf.dropbox.com:443/
2016-01-19 23:12:19.040 00000258 Request: www.google.com:443/



2016-01-20 09:35:34.135 7fff78fb5000 Info: Privoxy version 3.0.24
2016-01-20 09:35:34.135 7fff78fb5000 Info: Program name:
/usr/local/sbin/privoxy
2016-01-20 09:35:34.141 7fff78fb5000 Info: Loading filter file:
/usr/local/etc/privoxy/default.filter
2016-01-20 09:35:34.181 7fff78fb5000 Info: Loading filter file:
/usr/local/etc/privoxy/user.filter
2016-01-20 09:35:34.182 7fff78fb5000 Info: Loading actions file:
/usr/local/etc/privoxy/match-all.action
2016-01-20 09:35:34.183 7fff78fb5000 Info: Loading actions file:
/usr/local/etc/privoxy/default.action
2016-01-20 09:35:34.209 7fff78fb5000 Info: Loading actions file:
/usr/local/etc/privoxy/user.action
2016-01-20 09:35:34.218 7fff78fb5000 Info: Loading actions file:
/usr/local/etc/privoxy/regression-tests.action
2016-01-20 09:35:34.232 7fff78fb5000 Error: block action without reason
found. This may become a fatal error in future versions.
2016-01-20 09:35:34.235 7fff78fb5000 Info: Listening on port 8118 on IP
address 127.0.0.1
2016-01-20 09:35:35.603 700000081000 Request: api.simperium.com:443/
2016-01-20 09:35:35.857 700000104000 Request: api.simperium.com:443/
2016-01-20 09:35:39.626 700000187000 Request: notify3.dropbox.com:443/
2016-01-20 09:36:16.022 70000020a000 Request: www.google.com:443/


Ian
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
I've fixed my build environment so an OS X release is back on schedule.
Great.
I spoke slightly too soon.
Whilst I have a working binary it seems that the version I have of
Apple's own package building tool no longer runs on my system. Apple's
monolithic approach to dev tool distribution means I cannot upgrade it
without risking the loss of down-level target SDKs. I am therefore
evaluating a free alternative and have high hopes of transitioning to it.
At this rate I'll be a few days behind the curve, but I don't see a
problem with that.
Post by Fabian Keil
Post by Ian Silvester
One item of note is that Apple has replaced gcc with LLVM/clang, which
errlog.c:429:18: warning: absolute value function 'abs' given an
argument of type 'long' but has parameter of type 'int' which may
cause truncation of value [-Wabsolute-value]
this_thread = abs(this_thread % 1000);
^
errlog.c:429:18: note: use function 'labs' instead
this_thread = abs(this_thread % 1000);
^~~
labs
Clearly the sizes of long and int are platform dependent, but I don't
know if the code change it is suggesting is viable.
Given that the code is in an "#ifdef __MACH__" block
I didn't look into the source file, assuming it was a different level
of strictness from LLVM; clearly I should have done!
Post by Fabian Keil
I have
no strong opinions on how to deal with the warning and if
it should happen before the release.
While using labs() seems viable, it's not obvious to me that the
whole line is (still) useful. If it isn't, removing it may be an
even better idea.
Could you try that and check how the thread ids in the log messages change?
Okay, will do.
Ian
Post by Fabian Keil
Fabian
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Ijbswa-developers mailing list
https://lists.sourceforge.net/lists/listinfo/ijbswa-developers
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Fabian Keil
2016-01-21 16:46:54 UTC
Permalink
Post by Ian Silvester
I have tested removal of the get_thread_id MACH IFDEF. It is indeed
strictly unnecessary since even without it the threads are still
uniquely identified. I propose however that it is kept and modified as
per the compiler warning, since the raw IDs use four extra bytes which
could better be used by the remainder of the log line, plus the comment
in the code is strictly true.
Let me know if anyone disagrees.
I don't disagree with your labs() commit which seems to be
the best solution before the release.

My main objection to the MACH section in general is that
"this_thread % 1000" actually increases the chances of id
collisions and that it's only done on MACH while the issue
it's supposed to address exists on other platforms as well.

Eventually it may make sense to figure out a hashing scheme
that works cross-platform and is less likely to cause collisions.

Fabian
Ian Silvester
2016-01-21 17:24:05 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
I have tested removal of the get_thread_id MACH IFDEF. It is indeed
strictly unnecessary since even without it the threads are still
uniquely identified. I propose however that it is kept and modified as
per the compiler warning, since the raw IDs use four extra bytes which
could better be used by the remainder of the log line, plus the comment
in the code is strictly true.
Let me know if anyone disagrees.
I don't disagree with your labs() commit which seems to be
the best solution before the release.
My main objection to the MACH section in general is that
"this_thread % 1000" actually increases the chances of id
collisions and that it's only done on MACH while the issue
it's supposed to address exists on other platforms as well.
Eventually it may make sense to figure out a hashing scheme
that works cross-platform and is less likely to cause collisions.
Unless you know more than what is said in the comment, I read it to say
that the issue has only been shown to affect OS X, but probably affects
other OSes that use the MACH kernel.

Are you certain that collisions could occur as-is? It seems a remote
possibility to me.

If you are sure, perhaps rather than add the better hashing to our
already insanely long TODO list I simply remove the labs function call?
This would leave us with guaranteed-unique 9 byte long thread ids
resulting from having their rightmost three 0s trimmed. These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?

Ian
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Fabian Keil
2016-01-21 18:48:13 UTC
Permalink
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
I have tested removal of the get_thread_id MACH IFDEF. It is indeed
strictly unnecessary since even without it the threads are still
uniquely identified. I propose however that it is kept and modified as
per the compiler warning, since the raw IDs use four extra bytes which
could better be used by the remainder of the log line, plus the comment
in the code is strictly true.
Let me know if anyone disagrees.
I don't disagree with your labs() commit which seems to be
the best solution before the release.
My main objection to the MACH section in general is that
"this_thread % 1000" actually increases the chances of id
collisions and that it's only done on MACH while the issue
it's supposed to address exists on other platforms as well.
Eventually it may make sense to figure out a hashing scheme
that works cross-platform and is less likely to cause collisions.
Unless you know more than what is said in the comment, I read it to say
that the issue has only been shown to affect OS X, but probably affects
other OSes that use the MACH kernel.
Are you certain that collisions could occur as-is? It seems a remote
possibility to me.
Unless I miss something, after the "% 1000" there can't be more than
1000 unique values left and as a result there must be collisions if
there are more than 1000 different threads ids.

In practice the number of required threads to get collisions is probably
a lot lower, as the thread ids aren't assigned randomly or by incrementing
the previous thread id by one.

I suspect that the only guarantee is that two threads that are running
at the same time have different thread ids.
Post by Ian Silvester
If you are sure, perhaps rather than add the better hashing to our
already insanely long TODO list I simply remove the labs function call?
The change from abs() to labs() should have no affect on the collision
"risk".

BTW, it seems like we may be able to solve the issue by using
pthread_getthreadid_np() instead of pthread_self() in the future.
Post by Ian Silvester
This would leave us with guaranteed-unique 9 byte long thread ids
resulting from having their rightmost three 0s trimmed.
I don't think so.
Post by Ian Silvester
These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.

Fabian
David Schmidt
2016-01-21 18:58:25 UTC
Permalink
Post by Fabian Keil
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.
I wrote that comment. :-) 'debuggable' literally meant just that. If
you wanted to attach gdb, for example, you'd want to know which process
ID to attach to... but of course we're talking about threads, not
processes. So debuggability is actually moot here.

The idea was simply to make the thread ID look somewhat presentable as a
number, and still have a reasonable chance of not colliding with another
similarly truncated thread ID (in Privoxy). The Mach kernel has a very
chatty thread ID, so I was just trying to reduce clutter in the log.

- David
Ian Silvester
2016-01-21 19:57:53 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
I have tested removal of the get_thread_id MACH IFDEF. It is indeed
strictly unnecessary since even without it the threads are still
uniquely identified. I propose however that it is kept and modified as
per the compiler warning, since the raw IDs use four extra bytes which
could better be used by the remainder of the log line, plus the comment
in the code is strictly true.
Let me know if anyone disagrees.
I don't disagree with your labs() commit which seems to be
the best solution before the release.
My main objection to the MACH section in general is that
"this_thread % 1000" actually increases the chances of id
collisions and that it's only done on MACH while the issue
it's supposed to address exists on other platforms as well.
Eventually it may make sense to figure out a hashing scheme
that works cross-platform and is less likely to cause collisions.
Unless you know more than what is said in the comment, I read it to say
that the issue has only been shown to affect OS X, but probably affects
other OSes that use the MACH kernel.
Are you certain that collisions could occur as-is? It seems a remote
possibility to me.
Unless I miss something, after the "% 1000" there can't be more than
1000 unique values left and as a result there must be collisions if
there are more than 1000 different threads ids.
In practice the number of required threads to get collisions is probably
a lot lower, as the thread ids aren't assigned randomly or by incrementing
the previous thread id by one.
I suspect that the only guarantee is that two threads that are running
at the same time have different thread ids.
Post by Ian Silvester
If you are sure, perhaps rather than add the better hashing to our
already insanely long TODO list I simply remove the labs function call?
The change from abs() to labs() should have no affect on the collision
"risk".
BTW, it seems like we may be able to solve the issue by using
pthread_getthreadid_np() instead of pthread_self() in the future.
Post by Ian Silvester
This would leave us with guaranteed-unique 9 byte long thread ids
resulting from having their rightmost three 0s trimmed.
I don't think so.
Post by Ian Silvester
These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.
Fabian
I wrote that comment. 'debuggable' literally meant just that. If
you wanted to attach gdb, for example, you'd want to know which process
ID to attach to... but of course we're talking about threads, not
processes. So debuggability is actually moot here.

The idea was simply to make the thread ID look somewhat presentable as a
number, and still have a reasonable chance of not colliding with another
similarly truncated thread ID (in Privoxy). The Mach kernel has a very
chatty thread ID, so I was just trying to reduce clutter in the log.

- David



Thanks for the input David!

So the upshot is that we don't need to worry about debuggability (I'll
modify the comment appropriately) and only need to ensure no numbering
collision whilst indeed making the log more readable.

Fabian, given the style of the raw ids (e.g. 7fff78fb5000) the '% 1000'
has the effect of simply stripping the least significant three digits,
which are always 000. This alone then does not affect the uniqueness of
the IDs. e.g. 7fff78fb5000 becomes 7fff78fb5, right?

Since they remain unique at that stage, taking their absolute value as
long ints will still give a unique value - I think we are safe here with
the abs->labs change. All that needs to change is the comment ;o)

Unfortunately pthread_getthreadid_np() is not available on OS X. Here we
have pthread_threadid_np() instead which I cannot recommend, both
because the syntax is significantly different and because it is only
available since 10.6, which'd cut off OS X versions we currently support.

Ian
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Fabian Keil
2016-01-22 13:03:11 UTC
Permalink
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.
[...]
Post by Ian Silvester
I wrote that comment. 'debuggable' literally meant just that. If
you wanted to attach gdb, for example, you'd want to know which process
ID to attach to... but of course we're talking about threads, not
processes. So debuggability is actually moot here.
The idea was simply to make the thread ID look somewhat presentable as a
number, and still have a reasonable chance of not colliding with another
similarly truncated thread ID (in Privoxy). The Mach kernel has a very
chatty thread ID, so I was just trying to reduce clutter in the log.
- David
Thanks for the input David!
So the upshot is that we don't need to worry about debuggability (I'll
modify the comment appropriately) and only need to ensure no numbering
collision whilst indeed making the log more readable.
Fabian, given the style of the raw ids (e.g. 7fff78fb5000) the '% 1000'
has the effect of simply stripping the least significant three digits,
which are always 000. This alone then does not affect the uniqueness of
the IDs. e.g. 7fff78fb5000 becomes 7fff78fb5, right?
Nope:

***@r500 ~ $perl -e 'printf("%x\n", 0x7fff78fb5000 % 1000);'
3c8

To strip the least significant three hexadecimal digits, which indeed
would not change the uniqueness if they are always zero, you would have
to divide by 16 to the power of three:

***@r500 ~ $perl -e 'printf("%x\n", 0x7fff78fb5000 / 16**3);'
7fff78fb5
Post by Ian Silvester
Since they remain unique at that stage, taking their absolute value as
long ints will still give a unique value - I think we are safe here with
the abs->labs change. All that needs to change is the comment ;o)
Unfortunately pthread_getthreadid_np() is not available on OS X. Here we
have pthread_threadid_np() instead which I cannot recommend, both
because the syntax is significantly different and because it is only
available since 10.6, which'd cut off OS X versions we currently support.
That's good to know.

Fabian
Ian Silvester
2016-01-22 13:26:46 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.
[...]
Post by Ian Silvester
I wrote that comment. 'debuggable' literally meant just that. If
you wanted to attach gdb, for example, you'd want to know which process
ID to attach to... but of course we're talking about threads, not
processes. So debuggability is actually moot here.
The idea was simply to make the thread ID look somewhat presentable as a
number, and still have a reasonable chance of not colliding with another
similarly truncated thread ID (in Privoxy). The Mach kernel has a very
chatty thread ID, so I was just trying to reduce clutter in the log.
- David
Thanks for the input David!
So the upshot is that we don't need to worry about debuggability (I'll
modify the comment appropriately) and only need to ensure no numbering
collision whilst indeed making the log more readable.
Fabian, given the style of the raw ids (e.g. 7fff78fb5000) the '% 1000'
has the effect of simply stripping the least significant three digits,
which are always 000. This alone then does not affect the uniqueness of
the IDs. e.g. 7fff78fb5000 becomes 7fff78fb5, right?
3c8
To strip the least significant three hexadecimal digits, which indeed
would not change the uniqueness if they are always zero, you would have
7fff78fb5
Yes, apologies. I had a moment of clarity as I fell asleep last night
and realised I'd have to eat crow (as they say in North America) this
morning! Your post beat me to my direct apology.

We're tagged now so I won't make any change yet, but unless anyone
prefers otherwise I suggest we go with the division by 16**3 and drop
the absolute function - the ids are not pretty but at least they're
guaranteed unique.

Cheers,

Ian
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Ian Silvester
2016-01-22 13:26:55 UTC
Permalink
Post by Fabian Keil
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
These may still
not be unique in their first 4 bytes (though they're significantly more
different than the raw ids); I'll admit I don't understand the comment's
talk of a "debuggable value in the first 4 bytes" - do you think it
simply means a unique value in those 4 bytes? And what is so important
about the first 4 bytes of the value from a debugging perspective?
While I'm having trouble parsing the comment as well, I interpret it similarly.
I assume David meant to write "discernible" or "distinguishable" or something
like this and was using a 32 bit system which is relevant for the "4 bytes" part.
[...]
Post by Ian Silvester
I wrote that comment. 'debuggable' literally meant just that. If
you wanted to attach gdb, for example, you'd want to know which process
ID to attach to... but of course we're talking about threads, not
processes. So debuggability is actually moot here.
The idea was simply to make the thread ID look somewhat presentable as a
number, and still have a reasonable chance of not colliding with another
similarly truncated thread ID (in Privoxy). The Mach kernel has a very
chatty thread ID, so I was just trying to reduce clutter in the log.
- David
Thanks for the input David!
So the upshot is that we don't need to worry about debuggability (I'll
modify the comment appropriately) and only need to ensure no numbering
collision whilst indeed making the log more readable.
Fabian, given the style of the raw ids (e.g. 7fff78fb5000) the '% 1000'
has the effect of simply stripping the least significant three digits,
which are always 000. This alone then does not affect the uniqueness of
the IDs. e.g. 7fff78fb5000 becomes 7fff78fb5, right?
3c8
To strip the least significant three hexadecimal digits, which indeed
would not change the uniqueness if they are always zero, you would have
7fff78fb5
Yes, apologies. I had a moment of clarity as I fell asleep last night
and realised I'd have to eat crow (as they say in North America) this
morning! Your post beat me to my direct apology.

We're tagged now so I won't make any change yet, but unless anyone
prefers otherwise I suggest we go with the division by 16**3 and drop
the absolute function - the ids are not pretty but at least they're
guaranteed unique.

Cheers,

Ian
--
My PGP public key
<http://diem.serveftp.net:8080/IanSilvesterPGPPublicKey.asc>.
Fabian Keil
2016-01-24 18:21:37 UTC
Permalink
Post by Ian Silvester
Post by Fabian Keil
Post by Ian Silvester
Fabian, given the style of the raw ids (e.g. 7fff78fb5000) the '% 1000'
has the effect of simply stripping the least significant three digits,
which are always 000. This alone then does not affect the uniqueness of
the IDs. e.g. 7fff78fb5000 becomes 7fff78fb5, right?
3c8
To strip the least significant three hexadecimal digits, which indeed
would not change the uniqueness if they are always zero, you would have
7fff78fb5
Yes, apologies. I had a moment of clarity as I fell asleep last night
and realised I'd have to eat crow (as they say in North America) this
morning! Your post beat me to my direct apology.
We're tagged now so I won't make any change yet, but unless anyone
prefers otherwise I suggest we go with the division by 16**3 and drop
the absolute function - the ids are not pretty but at least they're
guaranteed unique.
Sounds good to me.

Fabian

Fabian Keil
2016-01-22 12:25:37 UTC
Permalink
Post by Fabian Keil
Post by Lee
Post by Fabian Keil
Post by Ian Silvester
IIRC, last time around Sourceforge's SVN tagging was broken - on the
offchance that is still the case, please let us know when you consider
the source is 'frozen' for this release.
Will do.
I just pushed the ChangeLog and documentation updates for 3.0.24
and intend to tag the tree on Friday without any changes except for
typo-fixes etc.
Does "tag the tree" include making the 3.0.24 source tarball? If no,
would you also do that?
I consider it a separate step, but intend to do it on Friday as well.
CVS is tagged and I was allowed to upload the source tarball myself
this time.

Fabian
Loading...