Comments (10)
OK I will try to fix 64-bit big-endian and see what I can do with 32-bits. I don't have machines to test on 32-bits. filearray
does assume 64-bits because just like package bit64
, it's using R 64-bit double
to store integer64
, which allows us to index seriously large tensor arrays (2^31 is too small for indexing). As long as package bit64
works on 32-bit system, filearray
should. It's not doing anything else special.
In theory filearray
should work on big-endian systems. However, when I flipped the internal endian sign, I immediately received errors, that means I was wrong and endianess is not handled properly.
from filearray.
@dipterix You have fixed everything:
R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(testthat)
> library(filearray)
R CMD check mode. Using 2 threads.
>
> # Sys.setenv("FILEARRAY_SKIP_COLLAPSE" = "TRUE")
>
> cat(utils::capture.output({
+ print(Sys.getenv())
+ }), sep = "\n", file = stderr())
BIBINPUTS .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib:
BSTINPUTS .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst:
CCACHE_DIR /opt/local/var/macports/build/.ccache
COLUMNS 80
COMMAND_MODE legacy
DEVELOPER_DIR /Developer
DISPLAY /tmp/launch-gQ9IMC/:0
DYLD_FALLBACK_LIBRARY_PATH
/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib
DYLD_LIBRARY_PATH /opt/local/lib/libgcc:/opt/local/lib/libgcc:/opt/local/lib/libgcc
EDITOR vi
FILEARRAY_NUM_THREADS 2
HOME /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home
LANG en_US.UTF-8
LANGUAGE C
LC_COLLATE C
LINES 24
LN_S ln -s
MAKE make
NO_PROXY *.local,169.254/16
PAGER /opt/local/bin/less
PATH /opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
PWD /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck/tests
R_ARCH
R_BATCH
R_BROWSER /usr/bin/open
R_BZIPCMD /opt/local/bin/bzip2
R_CMD /opt/local/Library/Frameworks/R.framework/Resources/bin/Rcmd
R_DEFAULT_PACKAGES
R_DOC_DIR /opt/local/Library/Frameworks/R.framework/Resources/doc
R_ENVIRON
R_ENVIRON_USER
R_GZIPCMD /opt/local/bin/gzip
R_HOME /opt/local/Library/Frameworks/R.framework/Resources
R_INCLUDE_DIR /opt/local/Library/Frameworks/R.framework/Resources/include
R_LIBS /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck
R_LIBS_SITE /opt/local/Library/Frameworks/R.framework/Resources/site-library
R_LIBS_USER /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home/Library/R/Power
Macintosh/4.3/library
R_OSTYPE unix
R_PAPERSIZE letter
R_PAPERSIZE_USER letter
R_PDFVIEWER /usr/bin/open
R_PLATFORM powerpc-apple-darwin10.0.0d2
R_PRINTCMD lpr
R_PROFILE
R_PROFILE_USER
R_RD4PDF times,hyper
R_SESSION_TMPDIR /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp/RtmpEuO8rp
R_SHARE_DIR /opt/local/Library/Frameworks/R.framework/Resources/share
R_STRIP_SHARED_LIB strip -x
R_STRIP_STATIC_LIB strip -S
R_SYSTEM_ABI macos,gcc,gxx,gfortran,gfortran
R_TESTS startup.Rs
R_TEXI2DVICMD /opt/local/bin/texi2dvi
R_UNZIPCMD /opt/local/bin/unzip
R_VERSION 4.3.0
R_ZIPCMD /opt/local/bin/zip
SED /usr/bin/sed
SHLVL 5
TAR /opt/local/bin/gtar
TEXINPUTS .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex:
TMPDIR /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp
USER root
_R_CHECK_INTERNALS2_ 1
_R_CHECK_LICENSE_ TRUE
_R_CHECK_PACKAGE_NAME_
filearray
_R_SHLIB_BUILD_OBJECTS_SYMBOL_TABLES_
TRUE
__CF_USER_TEXT_ENCODING
0x0:0:0
>
> test_check("filearray")
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]
══ Skipped tests (3) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-collapse.R:85:5', 'test-collapse.R:196:5',
'test-collapse.R:308:5'
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]
>
> filearray:::clear_cache()
>
> proc.time()
user system elapsed
70.335 6.858 79.813
Thank you!
from filearray.
The fix 0.1.6 has been submitted to CRAN
from filearray.
By the way, also OpenMP detection does not work: “OpenMP not detected. Using single thread only”. GCC does support OpenMP, however.
from filearray.
Oh wow. Thanks a lot for reporting this bug. I'll look into this. Would you mind showing me what can I do to reproduce this error? Also if this is too hard, I would be really thankful if you could help me test the devel version on your machine as I might have fixed fmap
functions.
Also the devel version of filearray
is not using OpenMP
anymore. It's using built-in tinythread Instead.
Thanks!
from filearray.
@dipterix Thank you for responding to the issue.
From the latest commit tests results are: [ FAIL 253 | WARN 11 | SKIP 3 | PASS 672 ]
I will attach a log now, too big to quote.
UPD.
filearray_34c7dcf1223dcc94a7204a465c78516d2820dc92.txt
Any ideas?
from filearray.
@dipterix I suppose that errors which look very wrong may be due to ignored endianness.
Also, does it use long doubles? PowerPC BE uses IBM format for it (same for macOS, AIX, *BSD, Linux – as long as those are Big-endian).
Re reproducing: I do not think these errors have anything to do with macOS specifics, so presumably this can be reproduced on any PowerPC platform, though 64-bit ones may have fewer failures. Common versions of BSD and some versions of Linux support PowerPC, both 32- and 64-bit. If no hardware is available, Qemu may be used, perhaps.
But I can test on my end whatever you suggest, that may be easier.
from filearray.
Thank you very much for working on this!
For sure, doubles are supported on 32-bit, and even 128-bit long doubles are (with a caveat of being composed of two doubles, IBM format). This should work in R: I believe, many packages are using these, and so far whatever we test on PowerPC, usually passes 100% tests or occasionally have rounding-related issues. I mean, the platform itself is perfectly functional, and all R-related software should work fine.
from filearray.
Hi @barracuda156 I just pushed a patch to the Github. Would you mind helping me test the devel version?
I think 64-bit should be fixed. I gue→↘↗ss, not so sure because I don't have big-endian machine by my hands. All I can do is to flip the endian flag in my code to force the file and machine to have different endianness.
Again, thanks a lot for helping me.
from filearray.
Awesome! It is already merged to Macports too: https://ports.macports.org/port/R-filearray/details
from filearray.
Related Issues (4)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from filearray.