Code Monkey home page Code Monkey logo

src's Introduction

Madagascar

CircleCI SourceForge codecov join slack

A package for reproducible geophysical data processing

http://www.ahay.org

What is Madagascar?

Madagascar is an open-source software package for multidimensional data analysis and reproducible computational experiments. Its mission is to provide

  • a convenient and powerful environment
  • a convenient technology transfer tool

for researchers working with digital image and data processing in geophysics and related fields. Technology developed using the Madagascar project management system is transferred in the form of recorded processing histories, which become "computational recipes" to be verified, exchanged, and modified by users of the system.

Design Principles

  • Madagascar is a modern package. Started in 2003 and publicly released in 2006, it was developed entirely from scratch following modern software engineering practices such as moduleencapsulation and test-driven development. A rapid development of a projectof this scope (more than 300 main programs and more than 3,000 tests) would not be possible without standing on the shoulders of giants and learning from the 30 years of previous experience in open packages such as SEPlib and Seismic Unix. We have borrowed and reimplemented functionality and ideas from these other packages.

  • Madagascar is a test-driven package. Test-driven development is not only an agile software programming practice but also a way of bringing scientific foundation to geophysical research that involves numerical experiments. Bringing reproducibility and peer review, the backbone of any real science, to the field of computational geophysics is the main motivation for Madagascar development. The package consists of two levels: low-level main programs (typically developed in the C programming language and working as data filters) and high-level processing flows (described with the help of the Python programming language) that combine main programs and completely document data processing histories for testing and reproducibility. Experience shows that high-level programming is easily mastered even by beginning students that have no previous programming experience.

  • Madagascar is an open-source package. It is distributed under the standard GPL open-source license, which places no restriction on the usage and modification of the code. Moreover, access to modifying the source repository is not controlled by one organization but shared equally among different developers. This enables an open collaboration among different groups spread all over the world, in the true spirit of the open source movement.

  • Madagascar uses a simple, flexible, and universal data format that can handle very large datasets but is not tied specifically to seismic data or data of any other particular kind. This "regularly sampled" format is borrowed from the traditional SEPlib. A universal data format allows us to share general-purpose data processing tools with scientists from other disciplines such as petroleum engineers working on large-scale reservoir simulations.

Where to get more information about Madagascar

The primary source of information is the web site:

http://www.ahay.org/

Additional information:

users' mailing list ("RSF-user"):

https://lists.sourceforge.net/lists/listinfo/rsf-user

developers' mailing list ("RSF-devel"):

https://lists.sourceforge.net/lists/listinfo/rsf-devel

development blog:

http://ahay.org/blog/

Compiling, Building, Installing and Testing

See the INSTALL.txt document for build instructions.

History

While being written from scratch, Madagascar borrows ideas from the design of SEPlib, a publicly available software package, maintained by Bob Clapp at the Stanford Exploration Project (SEP). Generations of SEP students and researchers contributed to SEPlib. Most important contributions came from Rob Clayton, Jon Claerbout, Dave Hale, Stew Levin, Rick Ottolini, Joe Dellinger, Steve Cole, Dave Nichols, Martin Karrenbach, Biondo Biondi, and Bob Clapp.

Madagascar was started, under the name RSF (Regularly Sampled Format) by Sergey Fomel in 2003. Since then, many other people have contributed to it. See the AUTHORS.txt file for an incomplete list.

src's People

Contributors

alexander-klokov avatar alhadab avatar cako avatar chenyk1990 avatar deckerla avatar dirack avatar ediaz avatar geogangfang avatar henryk-modzelewski avatar j5un avatar jenningsjwj avatar jgodwin avatar jshragge avatar kschleicher avatar maggiecopy avatar nutsprogrammer avatar perfrancesco avatar phamnam95 avatar psava avatar romanonly avatar sfomel avatar songxl2020 avatar sripanichy avatar symes avatar tharittk avatar yangliu1979 avatar yangpl avatar yunzhishi avatar zhichenggeng avatar zxue1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

src's Issues

non-standard keys in segyread/segywrite

Add the ability for sfsegyread/sfsegywrite/etc to store non-standard trace keys in addition to standard keys rather than remapping standard keys as it is done now.

Reported by: sfomel

bargraph

sfbargraph – a version of graph to plot bar plots,
could be used as an alternative to sfdots.

Reported by: sfomel

interactivity in xtpen

Add/restore interactivity options in xtpen.

Compare Madagascar’s xtpen with the version in SEPlib-5.0.

Reported by: sfomel

scalebar in sfgrey3

Using the option "scalebar=" in sfcubeplot triggers a "Need bar=" error. The scalebar should be made automatically, as with sfgrey.

Reported by: ivlad

segyread and SU files in XDR format

There is an outstanding problem (bug) with sfsegyread applied to reading SU files. It was discovered (the hard way) by Ivan Vasconcelos. As far as I understand it: if the user installs SU with XDR support, then the data in the SU files are stored in XDR (big endian) format while the headers remain in native format (little endian on Linux). There is no way to detect this from the file itself or to handle it with the existing endian= parameter. We might need to introduce an additional parameter to handle this particular case.

Reported by: sfomel

thplot enhancement

A surface plotting program, which is more enhanced than thplot.

Reported by: sfomel

Parallelize basic utilities with OMP

Parallelize basic utilities like sfattr and sftransp with OMP. For sfattr, test using a large dataset if it is worth the code duplication to move the tests for types outside the for loops.

Reported by: ivlad

Doctest in Python

Add doctest-style testing in Python API:
http://docs.python.org/library/doctest.html

Reported by: sfomel

pens documentation

Adding/restoring documentation for common pen parameters.

Requires editing init_vplot.c

Reported by: sfomel

scons zip

Add "scons zip" to rsftex to automate packaging a manuscript for submission to Geophysics. "scons zip" will include and replace "scons figs".

Reported by: sfomel

xtpen antialiasing

Joachim Mispel [email protected]
to rsf-user

I have a question about the display (on screen and ps output) of seismic data with xtpen (Xtpen). Whenever I try to display a dataset where the number of data points is (significantly) smaller than the number of pixels on the screen, the SEP/RSF image can look downright horrible; at least when compared with SU.

Reported by: sfomel

segyread enhancement

1. Alias suread to “segyread su=y”.
2. Allow it to accept stdin instead of tape=.
3. Fix SU XDR bug
http://sourceforge.net/tracker/index.php?func=detail&aid=1620414&group_id=162909&atid=825645

Reported by: sfomel

configure script

Most people installing software from source expect to run

./configure
make
make install

It would be good to write ./configure, a fairly verbose
shell script that would
1. check python installation
2. help the user assign environmental variables
3. check scons installation and install scons if necessary
4. run “scons config”
5. instruct the user to run “scons” and “scons install”

That should do it for users who are too lazy to read
the INSTALL file.

Reported by: sfomel

Guide to main programs

Complete the guide to main programs
http://egl.beg.utexas.edu/RSF/book/rsf/rsf/prog_html/

Programs missing from the guide:

rm
rotate
rtoc
scale
segyread
segywrite
spike
spray
stack
transp
window

Reported by: sfomel

selfdoc for pens

pen programs (vppen, xtpen, pspen, ppmpen) need proper selfdocs.

Reported by: sfomel

sfdatadoc

sfdatadoc would be a script analogous to sfdoc but documenting data from the public data repository
instead of programs.

sfdatadoc amoco

would provide a short description and list (in USED IN) all examples of using the Amoco dataset.

sfdatadoc -k synthetic

would list all available synthetic datasets

Reported by: sfomel

distributed file format and parallel flows

Implement mata-file format, useful for IO efficient parallel processing, as collection of RSF files existing on multiple systems in either local or shared directories.

Implement utilities for parallel flows.

Reported by: henrykm

man pages

Let sfdoc generate man/troff pages at compile time and put them in $RSFROOT/man (“man prog” would then work like “sfdoc prog” if one remembers to set $MANPATH).

Reported by: sfomel

graph3 completion

Complete sfgraph3 to be able to plot surfaces sideways
(orient=2 and orient=3) and with flat=n.

Reported by: sfomel

vppen outN=

vppen outN= option is not working properly (affects
vplot2gif)

Reported by: sfomel

unsigned char in XDR

Problem with unsigned char support in XDR. To reproduce:

sfspike n1=10 k1=5 | sfbyte | sfdisfil
0: 128 128 128 128 254 128 128 128 128 128

sfspike n1=10 k1=5 | sfdd form=xdr | sfbyte | sfdisfil
0: 128 128 0 0 0 0 0 0 0 0

Thanks to Antoine Guitton for reporting.

Reported by: sfomel

numpy interface

Update the python API from numarray to numpy.

Reported by: sfomel

Automatic routing around gcc 4.5 problem

Currently the configuration scripts prefer gcc. On systems with gcc 4.5 such as Fedora 14, this results in a compiler bug being triggered in shotfill.c. Possible solution:

Make the configuration scripts detect the gcc version, then, if version==4.5, do one of the following, or a combination thereof:

(1) Look for other compilers and give them precedence over gcc

(2) If no alternatives are found and distro==Fedora14, suggest package compat-gcc-34 and configuration with CC=gcc34, or package clang and configuration with CC=clang

(3) Possibly, put a placeholder instead of the programs using shotfill.c

Reported by: ivlad

Binary packages

In addition to the source release, it would be nice to
provide RPM, Debian, and Mac OS packages. Sourceforge
provides some pointers about software packaging:
http://sourceforge.net/docman/display_doc.php?docid=762&group_id=1#packaging

Jim Jennings has been looking at Mac OS packaging.

Reported by: sfomel

conjgrad extensions

sfconjgrad should be extended to handle more complicated
options: prior data model, regularization,
preconditioning, etc.

Reported by: sfomel

MEMSIZE environmental variable

Programs using memsize= parameter should get the
default memsize from a special environmental variable.

Reported by: sfomel

Java API

Add a Java API (possibly using SWIG).

Reported by: sfomel

minidds

Provided that Randy Selzler and BP agree to it and
release the code under a GPL-compatible license:

Separate the bridge program from DDS and all its
dependencies and create a minimalistic package. It
could be included in an RSF subdirectory and
compiled/configured with scons and also distributed as
a separate file.

Reported by: sfomel

Harlan's CG

Bill Harlan agreed to contribute his C++ conjugate-gradient code from
http://billharlan.com/pub/code/conjugate_gradients/
It needs to be integrated and tested in Madagascar fashion.

Reported by: sfomel

more "pens"

More “pens” for Vplot.

svgpen (to generate SVG)
pdfpen (PDF directly without going through Postscript)
gtkpen (better interface than xtpen)

Joe Dellinger writes:

If someone wants to jump in, start by reading the “vplothacker” documentation. It pretty well covers exactly what you have to know to write a new pen filter. That’s the documentation Dave used to write xtpen and Steve used to write pspen.

See http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/pens/docs/vplothacker.doc?view=markup

Reported by: sfomel

memory allocation in sftransp

Deli Wang writes:

> If I have a data : n1=6000,n2=300,n3=300
>
> I want to transpose it with “plane=13” .In the program the following lines
> -
>
> ```
> n1=sf_esize(in);
> n2=n3=1;
> for (i=0; i < dim; i++) { ****dim=3
> if (i < dim1-1) { ****dim1=1
> n1 = n[i]; / block size / *dim2=3
> } else if (i >= dim2) {
> n3 *= n[i]; /
loop over /
> } else {
> n2 *= n[i]; /
read n2 blocks at a time /
> }
> }
> ```
> -
>
> So,after that,
>
> ```
> n2=6000
300*300
>
> map = sf_intalloc (n2); **** still need to allocate a large memory in core
> ```

The program should be fixed so that map is stored out of core (in a temporary file) when n2 does not fit in memory.

Reported by: sfomel

vplotdiff

“scons test” does not work reliably between different
platforms because of floating-point round-off issues.

A special tool is needed to compare vplot files with
reasonable accuracy.

Possible inspiration is sfpldb and Reginald Beardsley’s
sucmp program from SU.

Reported by: sfomel

rsfbook completion

complete rsfbook.py – a python/scons module for
building books and reports with latex and latex2html.

Reported by: sfomel

segywrite extensions

Need the capability to generate SEGY headers and trace headers for segywrite.

Reported by: sfomel

lines in sfgrey3 with wantaxis=n

sfgrey should display lines identifying slices even
when wantaxis=n. The required change is in
plot/lib/stdplot.c

Reported by: sfomel

n-d histogram program (aka fold map)

Program that would take as input n 1-D RSF datasets and bin size parameters, and creates a n-D RSF output with histogram values. Can be used as histogram for n-tuples (or, for n=2, complex numbers histogram). Probably rough equivalent of SEPlib's Fold3D. It is important to allow integer inputs and outputs besides real-valued ones.

Reported by: ivlad

GUI: TKSU -> TKRSF

1) Add an option to sfdoc to output spec files in the
format that Jeff Thorson defined for TKSU. This would
make TKSU immediately applicable. Spec files can be
generated automatically at the compile time.

See http://www.henrythorson.com/interface.htm

2) Rewrite TKSU in Python using Tkinter.

For more discussion, see
http://sourceforge.net/forum/forum.php?thread_id=1579059&forum_id=552249

Reported by: sfomel

MPI launcher

A simple MPI program that would split the data into
chunks, send them to different nodes, run a system
command on each node, and collect the output.

Reported by: sfomel

Bezier curves in Vplot

Add Bezier curves as a vplot primitive. Then use them to port open-source Type 1 fonts to Vplot. (Joe Dellinger)

Reported by: sfomel

IRC channel

Create an IRC channel for real-life interaction among users and developers

Possibly use http://freenode.net/

Reported by: sfomel

planes in sfgrey3

Plotting like "sfgrey3 flat=n" but with intersecting planes instead of a cube. Suggested by Paul Sava.

Reported by: sfomel

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.