asutherland

9 lat temudiscuss

viz.js (https://github.com/mdaines/viz.js demo at http://viz-js.com/) is graphviz compiled to JS using emscripten. graphviz-d3-renderer (https://github.com/mstefaniuk/graph-viz-d3-js demo at http://graphviz.it/) uses viz.js to render to (x)dot format, then parse that so you can use d3 on top of it. Using those two, you should be able to leverage the layout smarts of graphviz with the presentation smarts of d3.

asutherland

10 lat temudiscuss

Clarification for those who parsed this (incorrectly) like I did: localStorage does work in Firefox in Private Browsing Mode[1], it's IndexedDB and the DOM Cache API that throw. (Although Firefox has private browsing support for IndexedDB in the works. That effort is tracked on https://bugzilla.mozilla.org/show_bug.cgi?id=781982.)

1: But it's exclusively memory-backed and will be forgotten somewhere between 0 and 20 seconds after the last active page for the origin is gone. It is not persistent for the lifetime of the private browsing session. However, because Firefox has a back-forward cache ("bfcache") that keeps pages around (frozen) for a while after you navigate the window to another location, the storage may be kept alive for some time even with no active windows.

asutherland

10 lat temudiscuss

If you're willing to try Firefox Nightly, check out the containers experiment: https://wiki.mozilla.org/Security/Contextual_Identity_Projec...

While everything still happens within a single profile, sites in different containers get different storage (cookies, localStorage, IndexedDB, etc.) and cannot see each other.

asutherland

10 lat temudiscuss

> So Mozilla is making it harder for end users to find and kill service workers?

about:serviceworkers was added as a hacky debugging/introspection tool with the expectation that the developer tools team would integrate the functionality into more full-featured, supported tooling. about:debugging is that tooling.

For example, about:debugging's "Workers" tab (on nightly): - Displays all registered Service Workers as well as whether they are stopped, enables starting them if desired, and attaching a debugger to them. - Also displays any open (Dedicated DOM) Worker instances as well as SharedWorker instances and provides debugging support. - Listens for changes and updates the UI in real-time covering whether the ServiceWorker is stopped/running/installing/etc. (about:serviceworkers renders in a single snapshot).

I'm parsing your underlying concern to be that ServiceWorkers are a powerful new capability for websites and that you want to make sure they're not being brushed under the rug as a "don't worry about it, nothing could possi-blye go wrong" (https://frinkiac.com/caption/S06E04/442491) situation.

It's indeed the case that ServiceWorkers are powerful and they open up new avenues of potential misbehavior. While the fundamental event-driven design of SW's where they're only spawned by explicit user action (browsing the site providing the service worker) or user-approved site actions (push notifications which require explicit user approval), there's still the potential for bad actors. Implementers share these concerns, which is where efforts like https://github.com/beverloo/budget-api and (potential? not sure if this is actually implemented?) heuristics like Chrome generating a (desktop) notification if a ServiceWorker gets woken up by a push and doesn't generate its own notification come from.

And I believe it's in these goals of having browsers proactively inhibit bad behavior and make users aware of the behavior of sites they have implicitly or explicitly granted permissions to, that the best defense of users is found. Which is to say, there will always be UI that helps users find and kill service workers, but the design goals are likely to center around a) debugging, b) performance/"what on earth is making my browser/computer run so slow!", or c) storage/"where did all my disk space go?" rather than one of manual gardening of service workers like they are weeds that spring up and users are responsible for cleaning up after.

asutherland

10 lat temudiscuss

2 quick hopefully informative nits:

It's better to consult the living standard of the editor's draft over the TR ("TR is for the TRash" as they say). The security section has been fleshed out a lot, for example: https://w3c.github.io/ServiceWorker/#security-considerations

In Firefox, about:serviceworkers is in the process of being replaced by about:debugging. The bug is https://bugzilla.mozilla.org/show_bug.cgi?id=1220747 if you want to follow-along, but start re-training your muscle memory now! :)

asutherland

10 lat temudiscuss

The SQLite numbers are not going to be realistic here given that "no SQLite transactions" are used and the write-ahead-log is probably not used. Assuming defaults are used, that means "PRAGMA synchronous=FULL" and "PRAGMA journal_mode=DELETE" (so no Write-Ahead-Log). This means every mutating statement is potentially going to result in multiple fsync() operations. See https://www.sqlite.org/pragma.html for context.

asutherland

10 lat temudiscuss

Here's the relevant Gecko (Firefox) code which tries to use the max-age and expires headers first and then will set it to forever if the response code was 300, 410, 301, or 308. Note that I'm going by a somewhat shallow code reading after a recent investigation. There's a lot of stuff going on in gecko/necko that could potentially apply some failsafe time limit, maybe in the cache implementation, so I wouldn't take this as 100% for sure. Breaking on this function with gdb and tracing the flow is probably a better idea if you really want to know.

https://dxr.mozilla.org/mozilla-central/source/netwerk/proto...

The 301, 308 stuff comes from IsPermanentRedirect which is here: https://dxr.mozilla.org/mozilla-central/source/netwerk/proto...

asutherland

11 lat temudiscuss

http://kangax.github.io/compat-table/es6/ and http://kangax.github.io/compat-table/es7/ are particularly good for ES since it breaks out things into the nitty gritty features, providing the code samples that are tested.

asutherland

11 lat temudiscuss

SQLite addressed writers blocking readers in 3.7.0 with its Write-Ahead Log. See https://www.sqlite.org/wal.html for more details, but point 2 at the top is "WAL provides more concurrency as readers do not block writers and a writer does not block readers. Reading and writing can proceed concurrently."

(Writes will still block writes of course.)

asutherland

11 lat temudiscuss

The flies add up. There's not a harassment noise floor beneath which we should ask the harassed to just shrug off harassment by their coworkers.

asutherland

11 lat temudiscuss

How would this be a witch hunt? The third definition from https://en.wiktionary.org/wiki/witch-hunt is "A public smear-campaign against an individual" which seems to better describe the actions of https://www.reddit.com/user/aoiyama. One could try to be pedantic about the second definition, "An attempt to find and publicly punish a group of people perceived as a threat, usually on ideological or political grounds.", but if you read the posts at the reddit link and the Mozilla Community Participation Guidelines at https://www.mozilla.org/en-US/about/governance/policies/part..., even if you don't agree with the Mozilla ideology, the aoiyama one is clearly incompatible with the Mozilla one.

If the only course of action is to say "Oh no! They used a throwaway account, so there's nothing we can do about the toxic environment posts like this create for members of our community!" that doesn't bode well for having a non-toxic environment. And since I do need to disclaim that I am a MoCo employee (but 100% speaking for myself alone), I should also mention that you will find in that list of posts a link to https://www.reddit.com/r/MensRights/comments/35u1yp/an_email... where the contents of a Mozilla Corporation internal-mailing list post were reposted. So even if one wanted to write this off as just a random internet troll, the situation is that there is either a MoCo employee intentionally harassing another MoCo employee and Mozillian or a MoCo employee passing information to someone doing the same thing.

I do want to be clear that I am not attributing these hypotheticals to your one-sentence reply. But I also want to be clear that we can't write off this type of toxic behaviour as acceptable because of the risk of being perceived as engaging in witch hunts/star chambers/other hyperbolic misapplications of known-bad-terms.

asutherland

11 lat temudiscuss

This then assumes that authentication is handled by some combination of device proximity, physically pushing a "grant admin access" button on the device, or falling back to password management (possibly on a sticker on the bottom of the device).

There is something to be said for tying the device into an existing strong authentication infrastructure.

asutherland

11 lat temudiscuss

Firefox's Places subsystem does use SQLite. Firefox's built-in SQLite implementation is compiled with the SQLITE_SECURE_DELETE mechanism referenced elsewhere in this thread (https://dxr.mozilla.org/mozilla-central/source/db/sqlite3/sr...) and a configure check is performed when Firefox is built with system SQLite (https://dxr.mozilla.org/mozilla-central/source/configure.in#...).

Additionally, there's a unit test that verifies that the feature is really working at https://dxr.mozilla.org/mozilla-central/source/storage/test/... that checks that a string shows up in the database and then disappears from the database when the row is removed from the database.

asutherland

11 lat temudiscuss

The release management page is https://wiki.mozilla.org/Release_Management/B2G_Landing. You can probably think of the code complete date as the lower bound on a production device shipping with that version of the software.

asutherland

11 lat temudiscuss

Assuming you're using the global search mechanism, a porter stemmer is used which means that the search engine sees all of "house", "houses", "housed", "housing", etc. the same.

The search also biases the results based on recency and things like whether the message has been starred/flagged, whether it was authored by/involves contacts in your address book, etc.

e15ctr0n is right that your best option is to use quotes to do a phrase search of "open house" in this situation, although you will still run afoul of the porter stemmer. (Unfortunately a post-pass filter if you realllly want "house" was never implemented. There is an open bug, however.)

asutherland

11 lat temudiscuss

> But that's really not the point anyway. Burying opt-out phone home behaviour in nothing but legalese small print is a dark pattern. Having no way to disable it without going into obscure parts of the UI that no normal user (or even normal power-user) is ever likely to find is also a dark pattern.

I agree that "Burying opt-out phone home behaviour in nothing but legalese small print is a dark pattern." But I think you're mis-characterizing this specific instance of the blocklist ping as "covert, opt-out surveillance" and the arguably fairly readable privacy policy as "legalese small print".

Specifically, I think the blocklist feature paragraph is quite good and not weasel-words. It explains:

- Tersely what/when/why Thunderbird does the blocklist ping: "Thunderbird also offers a Blocklist feature. With this feature, once a day Thunderbird does a regularly scheduled, automatic check to see if you have any harmful add-ons or plug-ins installed."

- What Thunderbird does with that information: "If so, this feature disables add-ons or plug-ins that Mozilla has determined contain known vulnerabilities or major user-facing issues or fatal bugs (e.g., Thunderbird crashes on startup or something causes an endless loop). You may view the current list of Blocklisted items."

- The information included in the blocklist ping: "This feature sends Non-Personal Information to Mozilla, including the version of Thunderbird you are using, operating system version, build ID and target, update channel, and your language preference. This feature also sends Potentially Personal Information to Mozilla in the form of your IP address and a cookie."

- What Mozilla does with the information (which is indeed not trivially obvious): "In addition, Mozilla also uses this feature to analyze Thunderbird usage patterns so we may improve our products and services, including planning features and capacity."

- A disclaimer about the lack of UI: "Currently there is no basic user interface to disable the Blocklist feature."

And then we have 2 more sentences:

- The link on disabling and why you wouldn't want to disable: "This feature can be disabled by following the instructions in this article. Disabling the Blocklist feature is not recommended as it may result in using extensions known to be untrustworthy."

And that was all of it.

In regards to the UI, if there had been a discussion about whether we should have a basic UI affordance for disabling the feature (there was not, to my knowledge), I think the bulk of the Thunderbird team would have argued against it because the risk to the user of rogue plugins/extensions was and continues to be serious. (Plugins probably more than extensions; Thunderbird tends to pick-up all the plugins that Firefox would see and most adware/malware implementors seemed otherwise unconcerned with Thunderbird.) Now if the checkbox also entirely disabled extensions and plugin loading, that could provide a safe trade-off for the user. But then we run into the whole "supported configuration problem". Every option adds new permutations that can lead to new failures, etc.

asutherland

11 lat temudiscuss

> Right, but the fact seems to remain that Thunderbird is phoning home in a way I can't obviously switch off (I use few extensions, and don't feel any need for such a blacklist) and that wasn't disclosed.

The privacy policy at https://www.mozilla.org/en-US/thunderbird/legal/privacy/ (which is linked to from the about dialog and perhaps other places as well) does explicitly call out the blocklist at https://www.mozilla.org/en-US/thunderbird/legal/privacy/#blo... and includes a link telling you how to disable it, https://support.mozilla.org/en-US/kb/thunderbird-makes-unreq....

Of course, you may have installed Thunderbird prior to the privacy policy existing in that form with those details. The specific privacy policy is from Oct 11, 2011 and the subversion log at http://viewvc.svn.mozilla.org/vc/projects/mozilla.org/trunk/... suggests it was a newish thing, although I would expect a privacy policy likely existed in other forms prior to that, but that's the limits of my subversion-fu.

asutherland

11 lat temudiscuss

> Also, it is more than a little creepy that Thunderbird is apparently phoning home every day to tell them I'm using it. Not cool, Mozilla.

Note that if you follow the blog post's link to https://wiki.mozilla.org/ADI it says "Mozilla measures Firefox usage by the number of Firefox installations that retrieve blocklist updates from Mozilla's servers each day."

The "blocklist" in turn links to https://wiki.mozilla.org/Blocklist which says the following at the top: "Blocklisting is the ability to disable errant add-ons, plugins, and other third-party software for Firefox users. For graphics drivers, please see this policy."

Note that Thunderbird also can send telemetry data to Mozilla for performance data reasons/etc. This is notified by an infobar at first-run or upgrade that says "Would you like to help improve Mozilla Thunderbird by automatically reporting memory usage, performance, and responsiveness to Mozilla? Learn More" and then has "Yes" and "No" buttons.

relevant disclaimers since this touches on privacy stuff: I work for the Mozilla Corporation. I also previously worked on Thunderbird.

asutherland

11 lat temudiscuss

There's several things you could be running into:

* Auto-compaction isn't working and so the .msf files are growing larger than they need to be. Thunderbird's per-folder metadata indices are stored in a weird texty append-only database format known as "mork" that you may have heard of, usually preceded or followed by some cursing. The file must be read completely into memory, and in many cases this ends up synchronously happening on the main thread. The upside to this is that the thread-pane is very fast and allows sorting on all columns, etc. Right-clicking on a folder and choosing "compact" will trigger compaction of the mork database as well as the offline cache (a per-folder mbox-style file where the downloaded messages are stored).

* Not leveraging Thunderbird's offline storage and proactive syncing of messages. If you bring up the account settings for the account, and look at the "Synchronization & Settings" tab, you want "Keep messages for this account on this computer" checked. You also want checkboxes by folders you care about in the list that pops up when you click the "Advanced..." button. With these enabled, Thunderbird will proactively download the message bodies to your computer, making them available for both offline access and rapid message display.

Note that if you only launch Thunderbird when you want to read your mail and immediately close it when done reading, you won't be getting much benefit from this and things may even go slower as you fight Thunderbird playing catch-up with your mail.

* You have a lot of of messages in the folders in question. The message file loading is O(messages) and so is sync without CONDSTORE enabled. (Thunderbird's CONDSTORE implementation regrettably has some bugs and was turned off recently. See https://bugzil.la/912216) Additionally, gmail's IMAP implementation can take some time to open an IMAP folder with a lot of messages because it needs to build a sequence number mapping and do other stuff.

asutherland

11 lat temudiscuss

In general I'd advise against using the MailNews portion of SeaMonkey if you're also using the profile for web browsing. (MailNews is the guts of Thunderbird.)

MailNews results in a non-trivial amount of main-thread I/O that is going to badly jank your browser experience. The good news is that the worst of it will be when using the MailNews-related UI like switching folders for display, but it's still going to have an impact.