I've been doing some spring cleaning, and as part of it rebuilding my aged OneZoom con

I'm guessing the remaining Perl s are ancient history nowadays, and if we don't

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Notes & to-do's post clean-install about oztree HOT 21 CLOSED

lentinj commented on September 27, 2024

Notes & to-do's post clean-install

from oztree.

Comments (21)

davidebbo commented on September 27, 2024 1

FWIW, I've been using 3.10 since I got started on OZ.

from oztree.

lentinj commented on September 27, 2024 1

I'm not sure what to upgrade here?

One of our NPM dependencies will be requiring the option to work, and will need bumping / removing.

from oztree.

hyanwong commented on September 27, 2024

Most of the dependencies in README.markdown aren't necessary for running OneZoom, even more so now we have the tree-build repo. I've never installed most of the Python & Perl dependencies. @hyanwong could we tidy this up?

I am going to try switching over to using the tree-build repo this weekend. I will try to tidy up the dependency list then.

from oztree.

davidebbo commented on September 27, 2024

I am going to try switching over to using the tree-build repo this weekend. I will try to tidy up the dependency list then.

Note that in the tree-build repo, the dependencies are resolved automatically via setup.cfg (see https://github.com/OneZoom/tree-build/blob/main/setup.cfg#L12-L19). So there is no manual step. Also, it's all Python at this point, with no Perl.

from oztree.

lentinj commented on September 27, 2024

I'm guessing the remaining Perl scripts are ancient history nowadays, and if we don't delete them then at least their dependencies don't need to be this prominent.

Note that pymysql or mysql-connector-python aren't dependencies of the site, as web2py bundles it's own copy. All the listed python dependencies are for things in OZprivate/ServerScripts/Utilities AFAICS. Maybe there should be a requirements.txt in that directory, and a README.md explaining how to install them? (A package like @davidebbo has done would be better, but I'm guessing that's too much for code that is probably very infrequently used).

from oztree.

davidebbo commented on September 27, 2024

The new tree-build code supersedes subtree_extract.pl and tree_and_meta_parser.pl (in OZprivate/ServerScripts/TreeBuild). There are various other Perl scripts scattered around that I'm not familiar with and can't comment on.

from oztree.

lentinj commented on September 27, 2024

@hyanwong According to README.markdown we assume Python 3.7, and the Gruntfile has python3.7 hard-coded in it. I'm guessing this assumption needs to move with the times a bit, 3.7 is about to fall off the security updates radar. Any preferences what to? Are the server(s) still on 3.7?

from oztree.

hyanwong commented on September 27, 2024

Good point. Let's move to something more modern. I'll ping you on slack. 3.8 is possible I think.

from oztree.

hyanwong commented on September 27, 2024

I guess we should just go with 3.10, to avoid too many further updates? I will try this now.

from oztree.

davidebbo commented on September 27, 2024

I've been on 3.10 without issues, so that's probably a safe bet. That being said, I see the latest stable is now 3.12, and the 3.11 release notes claim some notable perf improvements, so it may be worth exploring.

And as a side note, it would be nice to move OZtree to a virtual env, like we have for tree-build. It avoids having to install systemwide packages (which can cause conflicts, ...).

from oztree.

hyanwong commented on September 27, 2024

Yes, both true. I agree about venvs, and perhaps we should just bite the bullet and go for 3.12?

from oztree.

lentinj commented on September 27, 2024

perhaps we should just bite the bullet and go for 3.12?

Debian stable (which isn't that old atm) is only at 3.11. Whilst we don't actively use Debian, I'd be wary of assuming newer than stable without some justification.

venvs

A venv for OZtree will need to happen soon, when switching operating systems/python versions is the obvious time.

from oztree.

hyanwong commented on September 27, 2024

Ah, and I see FreeBSD 13.2 (which is what we use on the server) only has 3.8 by default, although I guess I can install this: https://www.freshports.org/lang/python311?

from oztree.

lentinj commented on September 27, 2024

I don't know much BSD, but presumably so.

from oztree.

hyanwong commented on September 27, 2024

These are the npm messages I get when I try a clean install on my laptop (OS X). I assume most of these can be ignored (anyway, the node modules are only used to compile static JS code, so are presumably not security risks)

(py311) yan@Yans-New-Air OZtree % npm --v
10.2.5
(py311) yan@Yans-New-Air OZtree % npm install              
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: Use your platform's native performance.now() and performance.timeOrigin.
npm WARN deprecated [email protected]: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm WARN deprecated [email protected]: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142

added 1215 packages, and audited 1426 packages in 43s

102 packages are looking for funding
  run `npm fund` for details

34 vulnerabilities (9 moderate, 23 high, 2 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues possible (including breaking changes), run:
  npm audit fix --force

from oztree.

hyanwong commented on September 27, 2024

To get grunt to work, I also needed to:

export NODE_OPTIONS=--openssl-legacy-provider

(see https://stackoverflow.com/questions/69692842/error-message-error0308010cdigital-envelope-routinesunsupported; npm audit fix --force did not work for me

from oztree.

lentinj commented on September 27, 2024

(anyway, the node modules are only used to compile static JS code, so are presumably not security risks)

Yes, we vendor very little, if any, Javascript into the client. And all the server-side javascript we do run is just for builds

export NODE_OPTIONS=--openssl-legacy-provider

Upgrading will be the way to solve this though. Again, not a vast security risk, but eventually this cheating option will disappear as they get bored of maintaining old OpenSSL builds.

from oztree.

hyanwong commented on September 27, 2024

export NODE_OPTIONS=--openssl-legacy-provider

Upgrading will be the way to solve this though. Again, not a vast security risk, but eventually this cheating option will disappear as they get bored of maintaining old OpenSSL builds.

I'm not sure what to upgrade here? npm is on 10.2.5.

from oztree.

hyanwong commented on September 27, 2024

Tidy up package.json, so that npm ci can be used instead

In #687 I have upgraded to webpack 5.0.0, and it seems to compile the site just fine: I haven't checked npm ci though.

I didn't assume python 3.7 in the above, installed python 3.9 and Grunt gets upset.

Also in #687 I installed it all using a conda install of python 3.11, and it seems fine, so I have changed the hardcoded python version in the Gruntfile to 3.11, on the assumption that we'll be able to install that new version on the new server soon.

from oztree.

lentinj commented on September 27, 2024

I haven't checked npm ci though.

npm ci is a simpler version of npm install. npm install will update transitive dependencies in package-lock.json if it wants to. npm ci OTOH refuses to do so, it just installs what's in package-lock.json.

So once npm install works without making a change to package-lock.json, npm ci should similarly be happy.

from oztree.

hyanwong commented on September 27, 2024

So with #687 I get

(py311) yan@Yans-New-Air OZtree % npm ci
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated @babel/[email protected]: 🚨 This package has been deprecated in favor of separate inclusion of a polyfill and regenerator-runtime (when needed). See the @babel/polyfill docs (https://babeljs.io/docs/en/babel-polyfill) for more information.
npm WARN deprecated [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
npm WARN deprecated [email protected]: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.

added 984 packages, and audited 1220 packages in 8s

145 packages are looking for funding
  run `npm fund` for details

7 vulnerabilities (4 moderate, 2 high, 1 critical)

I guess that's fine @lentinj : would you recommend that I try to update anything else (e.g. the ones that are mentioned there), or would this break stuff?

from oztree.

Notes & to-do's post clean-install about oztree HOT 21 CLOSED

Comments (21)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent