Code Monkey home page Code Monkey logo

Comments (10)

dipterix avatar dipterix commented on May 26, 2024 1

OK I will try to fix 64-bit big-endian and see what I can do with 32-bits. I don't have machines to test on 32-bits. filearray does assume 64-bits because just like package bit64, it's using R 64-bit double to store integer64, which allows us to index seriously large tensor arrays (2^31 is too small for indexing). As long as package bit64 works on 32-bit system, filearray should. It's not doing anything else special.

In theory filearray should work on big-endian systems. However, when I flipped the internal endian sign, I immediately received errors, that means I was wrong and endianess is not handled properly.

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024 1

@dipterix You have fixed everything:


R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(filearray)
R CMD check mode. Using 2 threads.
> 
> # Sys.setenv("FILEARRAY_SKIP_COLLAPSE" = "TRUE")
> 
> cat(utils::capture.output({
+     print(Sys.getenv())
+ }), sep = "\n", file = stderr())
BIBINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bib:
BSTINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/bibtex/bst:
CCACHE_DIR              /opt/local/var/macports/build/.ccache
COLUMNS                 80
COMMAND_MODE            legacy
DEVELOPER_DIR           /Developer
DISPLAY                 /tmp/launch-gQ9IMC/:0
DYLD_FALLBACK_LIBRARY_PATH
                        /opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib:/opt/local/Library/Frameworks/R.framework/Resources/lib
DYLD_LIBRARY_PATH       /opt/local/lib/libgcc:/opt/local/lib/libgcc:/opt/local/lib/libgcc
EDITOR                  vi
FILEARRAY_NUM_THREADS   2
HOME                    /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home
LANG                    en_US.UTF-8
LANGUAGE                C
LC_COLLATE              C
LINES                   24
LN_S                    ln -s
MAKE                    make
NO_PROXY                *.local,169.254/16
PAGER                   /opt/local/bin/less
PATH                    /opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
PWD                     /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck/tests
R_ARCH                  
R_BATCH                 
R_BROWSER               /usr/bin/open
R_BZIPCMD               /opt/local/bin/bzip2
R_CMD                   /opt/local/Library/Frameworks/R.framework/Resources/bin/Rcmd
R_DEFAULT_PACKAGES      
R_DOC_DIR               /opt/local/Library/Frameworks/R.framework/Resources/doc
R_ENVIRON               
R_ENVIRON_USER          
R_GZIPCMD               /opt/local/bin/gzip
R_HOME                  /opt/local/Library/Frameworks/R.framework/Resources
R_INCLUDE_DIR           /opt/local/Library/Frameworks/R.framework/Resources/include
R_LIBS                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/filearray-572cbcc7e82857a5e2d53a8a1473614aa7c6468f/filearray.Rcheck
R_LIBS_SITE             /opt/local/Library/Frameworks/R.framework/Resources/site-library
R_LIBS_USER             /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.home/Library/R/Power
                        Macintosh/4.3/library
R_OSTYPE                unix
R_PAPERSIZE             letter
R_PAPERSIZE_USER        letter
R_PDFVIEWER             /usr/bin/open
R_PLATFORM              powerpc-apple-darwin10.0.0d2
R_PRINTCMD              lpr
R_PROFILE               
R_PROFILE_USER          
R_RD4PDF                times,hyper
R_SESSION_TMPDIR        /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp/RtmpEuO8rp
R_SHARE_DIR             /opt/local/Library/Frameworks/R.framework/Resources/share
R_STRIP_SHARED_LIB      strip -x
R_STRIP_STATIC_LIB      strip -S
R_SYSTEM_ABI            macos,gcc,gxx,gfortran,gfortran
R_TESTS                 startup.Rs
R_TEXI2DVICMD           /opt/local/bin/texi2dvi
R_UNZIPCMD              /opt/local/bin/unzip
R_VERSION               4.3.0
R_ZIPCMD                /opt/local/bin/zip
SED                     /usr/bin/sed
SHLVL                   5
TAR                     /opt/local/bin/gtar
TEXINPUTS               .:.:/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex::/opt/local/Library/Frameworks/R.framework/Resources/share/texmf/tex/latex:
TMPDIR                  /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-filearray/R-filearray/work/.tmp
USER                    root
_R_CHECK_INTERNALS2_    1
_R_CHECK_LICENSE_       TRUE
_R_CHECK_PACKAGE_NAME_
                        filearray
_R_SHLIB_BUILD_OBJECTS_SYMBOL_TABLES_
                        TRUE
__CF_USER_TEXT_ENCODING
                        0x0:0:0
> 
> test_check("filearray")
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]

══ Skipped tests (3) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-collapse.R:85:5', 'test-collapse.R:196:5',
  'test-collapse.R:308:5'

[ FAIL 0 | WARN 0 | SKIP 3 | PASS 926 ]
> 
> filearray:::clear_cache()
> 
> proc.time()
   user  system elapsed 
 70.335   6.858  79.813 

Thank you!

from filearray.

dipterix avatar dipterix commented on May 26, 2024 1

The fix 0.1.6 has been submitted to CRAN

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024

By the way, also OpenMP detection does not work: “OpenMP not detected. Using single thread only”. GCC does support OpenMP, however.

from filearray.

dipterix avatar dipterix commented on May 26, 2024

Oh wow. Thanks a lot for reporting this bug. I'll look into this. Would you mind showing me what can I do to reproduce this error? Also if this is too hard, I would be really thankful if you could help me test the devel version on your machine as I might have fixed fmap functions.

Also the devel version of filearray is not using OpenMP anymore. It's using built-in tinythread Instead.

Thanks!

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024

@dipterix Thank you for responding to the issue.

From the latest commit tests results are: [ FAIL 253 | WARN 11 | SKIP 3 | PASS 672 ]
I will attach a log now, too big to quote.

UPD.
filearray_34c7dcf1223dcc94a7204a465c78516d2820dc92.txt

Any ideas?

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024

@dipterix I suppose that errors which look very wrong may be due to ignored endianness.

Also, does it use long doubles? PowerPC BE uses IBM format for it (same for macOS, AIX, *BSD, Linux – as long as those are Big-endian).

Re reproducing: I do not think these errors have anything to do with macOS specifics, so presumably this can be reproduced on any PowerPC platform, though 64-bit ones may have fewer failures. Common versions of BSD and some versions of Linux support PowerPC, both 32- and 64-bit. If no hardware is available, Qemu may be used, perhaps.

But I can test on my end whatever you suggest, that may be easier.

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024

Thank you very much for working on this!

For sure, doubles are supported on 32-bit, and even 128-bit long doubles are (with a caveat of being composed of two doubles, IBM format). This should work in R: I believe, many packages are using these, and so far whatever we test on PowerPC, usually passes 100% tests or occasionally have rounding-related issues. I mean, the platform itself is perfectly functional, and all R-related software should work fine.

from filearray.

dipterix avatar dipterix commented on May 26, 2024

Hi @barracuda156 I just pushed a patch to the Github. Would you mind helping me test the devel version?

I think 64-bit should be fixed. I gue→↘↗ss, not so sure because I don't have big-endian machine by my hands. All I can do is to flip the endian flag in my code to force the file and machine to have different endianness.

Again, thanks a lot for helping me.

from filearray.

barracuda156 avatar barracuda156 commented on May 26, 2024

Awesome! It is already merged to Macports too: https://ports.macports.org/port/R-filearray/details

from filearray.

Related Issues (4)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.