!*********************************************************************************** !* * !* F A M O U S * !* * !* Frequency Analysis Mapping On Unusual Sampling * !* - - - - - - * !* Program developed and written by * !* F. Mignard OCA/Cassiopee * !* Version 4.6 - December 2005 * !* * !* SEARCH OF A SET OF FREQUENCIES IN A DISCRETE TIMES SERIES * !* * ! ********************************************************************************** ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! Version Date: 05-12-05 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! Latest updates : this version replaces version 4.5 issued in July 2005 ! ! major changes 4.5 --> 4.6 : no need for the input data to be sorted ! : deletion of the Tuckey window (problem at low frequencies) ! : fine tuning of the tolerance in the SVD algorithm ! : new version of the filtering algorithm for the non significant lines ! ! See below : version history for details ! ! ! Contact : ! Francois Mignard ! Observatoire de la Cote d'Azur / CNRS ! Le Mont Gros ! BP 4229 ! Nice (France) ! Email: francois.mignard@obs-nice.fr ! ! ! ! ! -------- ! OVERVIEW ! -------- ! ! This package contains a library of Fortran-90 routines which permits the ! frequency analysis of a time series regularly or irregularly sampled. The library ! is supplemented by several drivers and test data, which may be run as is, or may ! be used as templates to start other developments. ! ! FAMOUS carries out the frequency analysis of a time signal, with any kind of ! sampling. It can be used in many different situations to find periods in a signal, ! to represent the results of a numerical integration in solar system dynamics as ! as quasi-periodic function and identify the frequencies, to obtain a convenient ! and compact expression of a signal containing many periodic lines, to remove the ! periodic or quasi-periodic components of a times series, etc. ! ! Usually time-series derived from observations are not regularly sampled ! (and may even be very irregularly sampled in the case of astronomy or Earth ! science) and the main feature of this software is to handle these samplings. ! However it works very nicely on regular sampling, providing better results ! than a standard FFT (accuracy in the frequencies, amplitudes and phases, ! evaluation of the uncertainty). ! ! ! The number of frequencies is arbitrary and a time dependance of ! the amplitude of each spectral line is allowed to generate a Poisson series. ! ! Basically the fitted model is as follows : ! ! y(t) :: C_0(t) + Sum_k [ C_k(t)*cos(2pi*nu_k*t) + S_k(t)*sin(2pi*nu_k*t)] (1) ! ! where C_0(t), C_k(t), S_k(t) are polynomial of time of a given degree, ! ! a + b*t +c*t^2 + ... + d*t^(degmax) ! ! with the maximum degree fixed by the user for each k, thus degmax = degmax(k). ! By setting degmax = 0 for each line one has the standard quasi periodic representation : ! ! y(t) :: C_0 + Sum_k [C_k*cos(2pi*nu_k*t) + S_k*sin(2pi*nu_k*t)] (2) ! ! where C_k (k=0..nfreq) and S_k (k=1..nfreq) are constants. ! ! FAMOUS determines all the unknowns of (1) : ! - the frequencies nu_k (k=1..nfreq) ! - the coefficients of the polynomials C_0(t), C_k(t) and S_k(t) (k=1..nfreq) ! ! Hence, the number of unknowns depends on the constraints set by the user in the program settings. ! The maximum number of lines is given by the user, but several filtering may occur in the ! program ending with a decomposition on a smaller number of lines. ! ! The data y(x) is read in a textfile as xx(k), yy(k) , k= 1..npts with the columns where ! xx(k) and yy(kk) lie are indicated in the program setting. ! The independent variable (call it the time) is not necessarily regularly sampled. ! From extensive experiments, I have not found a different level of reliability in the results ! according to the kind of sampling. However uniform sampling is inherently plagued by aliasing, meaning ! that there is not a unique representation over a set of frequency. On the other hand the noise outside ! the significant frequencies is usually much smaller than with an irregular signal, when the spectral ! window fills over a wide range of frequencies. But in this case the aliasing may become negligible ! as soon as the sampling becomes almost random. ! ! There are three main steps in the algorithm : ! ! Step 1 : Data reading and properties of the sampling and of the spectral contents of the window. ! ! Step 2 : Identification of a set of frequencies located with an accuracy of the order of ! sqrt(epsmach), [epsmach : machine precision] and then application of non linear ! least-squares fit to improve the location of the frequencies to epsmach. ! ! Step 3 : Filtering of the solution for non significant lines and ghost lines, statistics and ! general output. ! ! ! There are two major procedures called : ! ! FREQSEARCH performs step 1 and step 2 ! FREQSOL performs step 3 ! The main program is a driver that reads or simulates data, reads the ! settings (options for the run) and call the two procedures. ! It can be used as a frame to develop an application ! All the routines are included in a F90 Module called famouslib. ! ! Author : F. Mignard OCA/CNRS ! Observatoire de la Cote d'Azur ! Le Mont Gros ! BP 4229 ! francois.mignard@obs-nice.fr ! ! Acknowledgements : Several people provided useful test data or suggestions that have permitted ! to test and gradually improve the program. Working on someone else ! data proved very efficient for the program reliability, since ! one discovers at every trial unexpected shortcomings not seen with ! a simulation, always too ideal. ! ! I am grateful to : ! Ph. Berio, A. Bijaoui, C. Bizouard, G. Bourda, J. Chapront, ! L. Eyer, S. Lambert, M. Feissel, N. Houlié, G. Marcy, ! G. Metris, E. Slezak ! who contributed in some way (and quite often unknowingly) ! during the development phase, which extended over five years. ! Particular thanks are due to N. Houlié who provided detailed reports ! on problems he met and made valuable suggestions to improve the product. ! ! Thanks are also due to ! Brainerd, W.S., Goldberg, C.H. & Adams, J.C. (1990) "programmer's guide to ! Fortran 90", mcgraw-hill isbn 0-07-000248-7, pages 149-150 for the recursive ! quicksort routine ! ! and to ! Burton S. Garbow, Kenneth E. Hillstrom, Jorge J. More ! for the Minpack Project where I found the Levendberg-Marquardt fortran module ! (and the F90 version of A. Miller on his comprehensive personal repository at : ! http://users.bigpond.net.au/amiller/ ) ! ! All the other routines are mine. ! ! ! ! Legal matters for a plain human brain : ! ! The program is distributed freely and the author could not be responsible ! for any direct or indirect damageable consequence that might result from its use. ! ! ! The same for aliens : ! ! THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ! "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT ! LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ! A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT ! OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, ! SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT ! LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, ! DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ! THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT ! (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE ! OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ! ! ! Use and copies : ! You are authorised to distribute copies of this software without referring ! to the author. ! The author would appreciate, however, any reports of bugs ! or other difficulties that may be encountered. ! ! ! Reference : This work is original, although based on classical material. ! If you use it for your research work please refer to it to as : ! based on the frequency mapping FAMOUS, F. Mignard, OCA/CNRS, ! ftp://ftp.obs-nice.fr/pub/mignard/Famous ! !*********************************************************************************** ! ! -------------------- ! OVERALL ARCHITECTURE ! -------------------- ! The program calls two main subroutines, corresponding to the above steps. Many other ! procedures are used internally. ! ! Besides the input data file, a setting file is read with the various manual tuning ! needed. The default is the most automatic and several parameters can be disabled and ! replaced by user defined values or actions. ! ! For the most difficult case, a knowledge of the idiosyncrasies of the data help find the ! best parameters and to use the software on a trial and error basis by overriding the automatic ! settings. ! ! THE SETTINGS : input file setting.txt (with the standard driver) ! ------------ ! For each run on real data one needs to read a setting file with the following content : ! ! ! Settings of the program ! ! file_in ! Input filename with the data y(x) as xx, yy on each record ! icolx ! index of the column with the time data in file_in ! icoly ! index of the column with the observations in file_in ! file_out ! output filename ! numfreq ! search of at most numfreq lines ! flmulti ! multiperiodic (true) or periodic (false) search in the signal. ! flauto ! automatic search (true) or preset value (false) of the max and min frequencies ! frbeg ! preset min frequency in preset mode ! frend ! preset max frequency in preset mode ! fltime ! automatic determination (true) or preset value (false) of the time offset (used to regularise the algorithm : see comment below) ! tzero ! preset value of the origin of time if fltime = .false. in unit of the time signal read in the input file ! threshold ! threshold in S/N to reject non significant lines (< threshold) ! flplot ! flag for the auxiliary files (current power spectrum and remaining signal after k lines are found ) ! isprint ! control of printouts (0 : limited to results, 1 : short report, 2 : detailed report) ! iresid ! control the output of the residuals ! fldunif ! flag for the degree of the mixed terms (true : uniform degree for all terms) ! idunif ! degree if fldunif = .true. ! idegf(k) ! degree of each line if fldunif = .false. , k=0,numfreq ! ! An example : ! ! 'mydata.txt' input file with the data ! 1 field with time data in input file ! 3 field with observation data in input file ! 'mylines.txt' ouptut file with the frequencies and power ! 8 numf : number of lines searched ! .true. flmulti: frequency analysis of the signal as a multiperiodic time series. ! .false. flauto : flag for the frequency range .true. = automatic .false. : with preset frequencies ! 0.001d0 frbeg : lower frequency analysed in the periodograms (disabled if flauto = true) ! 8d0 frend : largest frequency analysed in the periodograms ! .true. fltime : flag for the determination of the time offset ! 0.d0 tzero : time offset with respect to the input data. (disabled if fltime = true) ! 3.5d0 Threshold in S/N to reject non statistically significant lines ! .false. Flag for the output of the periodograms and successive residuals (true = output) ! 0 iprint in freqsearch : 0 : setting and results, 1 : intermediate results, 2 : extended with correlations ! 0 iresid : = 1 residuals are printed. 0 : not printed ! .false. flagdeg : uniform degree for the mixed term (.true.) or not (.false.) ! 0 idm : Value of the degree in term like a +bt + ... ct^idm (used if flagdeg = .true.) ! 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ideg(k) k=0..numf loaded if flagdeg = .false. ! ! ! Comments : ! ! ! - file_in : name (or full path) of the input file with the data ! ! - icolx, icoly : columns where the data are. One must have icolx < icoly. ! ! - file_out : name (or full path) of the main output file with the results ! ! - numfreq : this gives the number of lines which the user wishes to project the signal on. ! due to filtering, the number may be smaller, although the full solution is ! printed in the output file before filtering. ! The right number depends on the signal, and some experimentation is needed ! to find where to stop. There is no general criteria that would fit every situation. ! In addition when the dynamics between the strongest and faintest lines is large, ! it may be advisable to introduce a time polynomial in the first few lines, to avoid ! the appearance of closely packed lines in the frequency domain. ! ! - flmulti : .true. : the programs searches for a signal representation on numfreq different spectral lines ! with unrelated frequencies. ! .false. : the program sets numfres = 2 and searches for a periodic signature. This applies typically ! to variable stars or radial velocity for extra-solar planets with irregular sampling. ! ! - flauto : .true. : the program determines the lowest and highest frequencies recoverable ! .false.: the user provides frbeg and frend for the range of the frequency search ! ! * for regular sampling :: no problem with the automatic mode. The largest ! frequency is found as the Nyquist frequency. However, it may be interesting ! to override this by using flauto = .false. to evidence the effect of aliasing and ! search in a another range of frequencies with a unique reprensation ! * for irregular sampling, there is strictly no maximum frequency. The program attempts to ! determines a value based on the distribution of the two point distance in the ! time sampling, by taking ordering the two-point time difference and taking the ! second decile to evaluate the equivalent of a Nyquist frequency. ! But this is an approximation, and again it is preferable that the user ! provides hints as to the frequency range. In addition, by letting the automatic mode go, ! the number of frequencies may be very large ( > 10^5 for example) yielding a large ! computing time, unecessary if one knows already in wich range to search. ! ! - frbeg,frend : lower and upper bound of the frequency range when flauto : .false. ! unit : inverse of the independant variable read in the data file ! ! - fltime : .true. : the program centers the time data about its mean value. That value is ouptut. ! .false. : the time offser is provided by the user in the setting. ! The algorithm is much more stable if the time data are nearly centered, with negative ! and positive values in similar number. The mean of the median perform quite as well (the mean ! is the default option. But for irrgular sampling this may produce an unpleasant number, while ! in the vicinity there is a significant round number. The user should select his own ! value with fltime = .false., provided it is within +/- 5% of the mean , scaled with the range. ! - tzero : when fltime : .false. this is the user choice for the time offset. ! then the data is offset as t_k <== t_k - tzero and tzero is printed in the final output file ! ! - threshold : lines found are rejected if their signal to noise (S/N) is found less than threshold. ! A value around 3 has been found reasonable, but the user needs to experiment with his ! own data, as the evaluation of S/N has nor firm statistical basis. ! What is done is the following : ! - one cannot use the residuals to estimate the significance of the lines, as the residuals ! after the k_th line depends of how much modelling remains there. In addition a real signal ! may have only a small fraction of its variance in a quasi-periodic signal. ! - the significance of a line is better appreciated in the frequency domain by looking ! on a plot whether the line appears as a well seen spike compared to its immediate surroudings. ! But for irregular samplings, the surroundings is not exactly noise, since we have the contribution ! to the spectrum of the observing window. So I have used a robust estimate of the remaining noise ! by looking at the spectra, after the line removal, and taking the median value. In this way ! all the contributions to the remaining spectral of other significant discrete lines, or that ! of the window itself are ignored and one can assume that one has a reasonable estimat of the ! contribution of the noise to the spectrum. Then one compute S/N as the ratio of the square root ! of the power in the line under consideration to the robust estimate of the noise. ! ! - flplot : flag to output at each frequency found the residual and the power spectrum. The ! files are called resid_k.txt and power_k.txt, where k : 0..nfreq and give the residuals ! signal and its power spectrum after the removal of the k_th line. For k=0 this means ! after the removal of the average ( when C_0(t) = C_0) or the polynomial trend when ! C_0(t) = a + b*t + c*t^2 ... where the maximum degree is determined with idegf(k) in the ! setting. This file are 2-columns spreadsheet with each record as (t_k, value_k) and ! are very useful for plotting. ! With a 'call system' inside the program (search the string 'system' to find the checked line) ! you can activate your favorite plotter interactively from FAMOUS or simply keep the files ! for an external plot. ! ! - iprint : control the extent of the output. This is not yet perfect, but just experiment to find ! which output is the best for you by selecting iprint = 0, 1 and 2 successively. ! ! - iresid : to output in the working directory the final residuals in a file called resid_final.txt. ! When there is a final filtering this is different from resid_nfreq since the residuals are ! computed from the last fitting. ! ! - fldunif : flag to control the degree of the polynomial of time. ! .true. : for any frequency (k=0..numfreq) the polynomials have all the same ! degree, given in the next record. In practice this should be used ! only to have deg(k) = 0 for all k, meaning that on look for a ! standard quasi-periodic model, without time dependance ! .false.: the degree of the polynomial can be chosen for each frequency ! and given in the second next record (here for at most 30 lines) ! ! - idm : if fldunif = true , idm is the degree common to all polynomials (usually idm = 0). ! ! - ideg(k) : each integral value in the last line of the setting file gives the degree ! of the time polynomials for each line (from 0 to numfreq) ! ! REMARKS : ! in the simulation mode (driver famous_simul.f90, the setting file is setting_simul.txt ! and the first three records with file_in, icolx, icoly are not present) ! !*************************************************************************************************** ! Comments on the standard output !*************************************************************************************************** ! When iprint = 1 one get a relatively detailed output with : ! ! - program version, exectution date and time ! ! - printing of the setting ! ! - properties of the time sampling in the time domain ! ** the last parameter is the Nyquist frequency computed with the average time step for an irregular sampling ! ! - properties of the time sampling in the frequency domain ! this includes : ! Line width in the frequency domain (~ spectral resolution) ! Stepsize in the frequency domain ( frequency sampling) ! Lowest frequency recoverable ! Typical highest freq. recoverable (or Nyquist for a regular sampling) ! First (lowest) frequency analysed ! Last (highest) frequency analysed ! Number of frequencies sampled ! ! - then there is an execution report frequency by frequency as the search progresses ! ! - final result before filtering out the non significant or ghost frequencies ! ! - final result after the filtering. ! ! The bottom of the file should be rather like : (with your numbers obviously) Frequency Period cos term sine term sigma(f)/f sigma(cos) sigma(sin) amplitude phase(deg) S/N 0 t^0 0.00000000E+00 1.49618818E+08 4.948E+01 1.496188E+08 0 t^1 0.00000000E+00 -1.06267041E-03 4.318E-03 -1.062670E-03 1 t^0 2.73776260E-03 3.65261766E+02 1.42809318E+06 -2.05114714E+06 1.151E-04 7.046E+01 1.048E+02 2.499331E+06 55.1527736 7.84E+03 t^1 -2.98658391E-01 -4.03762707E-03 4.060E+00 2.826E+00 2.986857E-01 179.2254532 2 t^0 5.47554277E-03 1.82630297E+02 7.26129331E+03 1.95372142E+04 8.505E-06 6.974E+01 7.024E+01 2.084296E+04 290.3883245 1.67E+02 3 t^0 3.38634692E-02 2.95303471E+01 -3.33208755E+03 3.27937835E+03 6.192E-06 6.999E+01 7.019E+01 4.675161E+03 224.5432253 9.08E+01 4 t^0 2.50786180E-03 3.98746055E+02 1.75373451E+03 1.61234893E+03 1.628E-04 7.044E+01 6.985E+01 2.382279E+03 317.4051803 5.71E+01 No line has been rejected from the significance test No line has been rejected from the spectral resolution test Execution time in seconds : 6.73 where the columns give : - line id ** one line is (A_0 + A_1*t +A_2*t^2+ ...)*cos(2pi*nu*t) + (B_0 + B_1*t +B_2*t^2+ ...)*sin(cos(2pi*nu*t) ** id = 0 stands for the zero frequency (~ average or time trend or best polynomial of larger degree) - k = power of time for the record - frequency in cy/unit of time - period = 1/freq in unit of time of the input data - A_k and B_k for each line and power of time - estimate of the relative accuracy of the frequency as sigma(nu)/nu - sigma(A_k) and sigma(B_k) - amplitude = sqrt(A_k^2 + B_k^2) for each power of time - phase with A*cos(2pi*nu*t) + B*sin(2pi*nu*t) ==> amp*cos(2pi*nu*t + phi) - S/N : signal to noise ratio of the line as found in the frequency domain (see comment at 'threshold' in the setting) !****************************************************************************************************** !****************************************************************************************************** ! ! Method Outline ! -------------- ! First step ! ---------- ! Data reading : basic settings are read in a file called 'setting.txt', whose content is detailed above. ! The data should be organised as a text file with one column for the time sequence (xx(:) or x(:) in the ! routines) and the data itself (yy(:) or y(:)) in the routines). This data may be in any columns of the ! file with the constraint col(x) < col(y). ! Then one determines the properties of the sampling, decides whether it is uniform or ! irregular. The time sequence is centered about its mean value (this gives a much better stability ! to the S/W and allows to compute realistic uncertainties for the frequencies and the amplitudes). ! The spectral resolution (width of a spectral line at half-maximum) is determined by generating ! several time sequence with y_k(t) = cos(2pi*nu_k*t) for about 10 relevant frequencies in the recoverable ! frequency range. ! For each frequency the spectral line is sampled at high resolution to determine the WHM. ! The spectral resolution is the average of this width_k. WHen the sampling is uniform, a single ! line is computed (and the result compared to the theoretical resolution). ! ! OUTPUT OF STEP 1 : sampling statistics in the time and frequency domain. In particular one attempts ! to determine the maximum frequency recoverable without aliasing. While this concept is rather simple ! for a regular sampling, nothing similar applies to an irregular, let alone a random sampling. In principle ! as soon as the sampling departs from uniformity one should be able to recover any frequency from the signal ! but in practice, this is very gradual and there is no good quantitative way to tell by how much as ! signal is not regular. Therefore for irregular sampling, it is preferable that the user tells ! the program (this is permitted in the setting) the range of frequencies where he wishes to search ! for a quasi-periodic signal. ! ! Second step ! ----------- ! The general idea follows the principles set forth by J. Laskar : Once k frequencies ! have been identified one removes from the signal as much as possible from this k lines and ! searches the next frequency in a space orthogonal to the space containing the k lines. ! However the implementation is totally original and specifically adapted to non ! regular sampling. ! ! One searches the most significant frequencies in y(x) with a periodogram . ! The minimum, maximum and stepsize frequencies are determined automatically ! from the the overall properties of x(k) : range, mean density, clumping ... ! The stepsize in the frequency domain is such that no line will be missed ! in the periodogram. ! No FFT is used in this phase to allow for irregular sampling, although optimised routines are ! called when a regular sampling is found (internally determined by the S/W) ! ! Once a line has been located a vernier is applied to determine its precise location ! and the power in the line. The vernier is based on a high resolution periodogram of ! limited range around the center of the line. ! The max is eventually located with a three point parabolic interpolation ! As the max is located to within the machine accuracy (epsmach), the abscissa (= frequency)) is ! at most known to sqrt(epsmach) in this step. Therefore a non-linear least-squares is applied to ! obtain the precise location of the frequence at epsmach. ! The level of significance of each line is evaluated from the periodogram by looking by how much the ! top of the line is different from the immediate surrounding in the frequency space. (see the comment ! at the field threshold in the description of the setting parameters above). ! ! The residuals of the fit with this model are by construction (this is a characteristic property of the least squares best fit) orthogonal to ! the linear space generated by the model. Therefore the residuals are now used as input to the search ! of the next frequency. ! The loop ends when numfreq lines have been found, with their ampllitudes and polynomial coefficients ! when relevant. ! ! ---------- ! Third step ! ---------- ! A final fit (non linear least squares) is performed with the highest accuracy given by ! tol = 1E-12 for the relative error and this solution is output. Then the non-significant lines ! are removed (when S/N < threshold) in one pass a new fit is performed on the new model , when at least ! one line is deleted. ! The second filtering is more complex and is called removal of ghost lines : ! when one removes from the signal a line, in principle the ! remaining signal is orthogonal to all the previous lines found. In practice, this is true for the frequency ! corresponding to the center of the line, and some small spurious lines in the wing of the main lines can ! be generated in this process and reappear latter, giving rise to pairs or triplets of lines of ! very similar frequency. From the point of view of the model fitting, there is nothing wrong there, but ! usually these pairs of lines have large amplitude and happen to be nearly out of phase, meaning that ! there is a difference cancellation. To keep the model more robust, with lines separated by at least ! half the width in the frequency space, one locate the pairs and removes the fainter of the two, before ! performing a new non-linear fit. This is done iteratively, since new pairs may show up in this process. ! It is important to note that two lines with delta(nu) < spectral resolution, can be represented either ! by one line of intermediate frequency, or much better by having the amplitude with a linear term in time : ! ! a*cos(2*pi*nu*t) + b*cos(2*pi*(nu'*t) ! ! with (nu'-nu)T/(2*pi) << 1 (T time range of the data) can be written as : ! ! (a+b)*cos(2*pi*nu*t) -2pi*b*(nu'-nu)*t*sin(2*pi*nu*t) ! ! and then can be easily represented by including a Poisson term of first degree in time. ! ! Then a linear least-squares is set up in order to determine the covariance matrix (the solution is not ! used) and to estimate the precision of the solution. This is scaled to the root-mean-square of the residuals ! so it depends on how much of the initial variance of the signal is removed in the model, rather than a ! measure of the random noise. At least this allows to produce an estimate of some uncertainty in the ! coefficients, that would be correct in the statistical in specific cases. ! The quality of the fitting is also measure by comparing the residuals from the linear solution to ! the actual residuals measured from the non-linear model. When everything goes well, the two are nearly ! identical. On the other hand when the system is linearly unstable, a difference shows up. ! WARNING : the time read in xx(k) is centered about its mean and the new origin is given in ! the ouptut. All the time functions are referred to this origin. ! ! ----------- ! ENVIRONMENT ! ----------- ! ! Normal mode : processing on a file of real data ! --------------------------------------------- ! ! The program environment is provided by the user in the file SETTING.TXT with : ! ! ! - input filename containing xx(k), yy(k) in each record ! - positions of the fields in this file ! - output filename for the various printouts ! - number of frequencies to be searched ! - setting flags for different running options to override defaults ! - printing options ! ! DRIVER : famous_driver.f90 ! ! By default the unit of time is taken from the sampling in xx(k) as the increase of ! xx of one. Then the unit of frequency is just the inverse. ! ! The module famouslib contains the common constantes, variables, arrays used by the libray ! and all the subroutines called by the driver to run the frequency mapping ! ! ! Simulation mode : tutorial for times series ! ------------------------------------------- ! ! The environment is read in SETTING_SIMUL.TXT : same as before without the first three records. ! ! DRIVER : famous_driver_simul.f90 ! ! Additional module : module simul_famous in the source file simul_famous.f90 ! ! Dedicated routines to simulate quasi periodic series this can be used to practice with FAMOUS ! but also to get insights with purposedly built series on the properties ! of the time sampling in frequency mapping. ! The module provided a framework to help the user develop his own simulations. ! ! There are three kinds of routines designed to generate various signals ! See the introduction in the source file. ! ! ! Multi file mode : sequence of data to be analysed. ! -------------------------------------------------- ! ! This third case has been developped specifically in the framework of the ESA space mission ! Gaia to analyse the observations of variable stars, but could be the starting point ! for applications with similar structure. The driver can read an input file ! containing successive times series in sequence. By setting the upper bound of the reading loop to ! ndata = 1, this is essentially the same as the first mode. ! A dedicated setting file is used : SETTING_PHOTOM.TXT ! no output of resiuals and power spectrum, but output of folded lightcurves if FLFOLD = .TRUE.. ! The reading loop is protected against an earlier end of file. ! ! DRIVER : famous_driver_photom.f90 ! ! !*********************************************************************************** !* Versions History * !*********************************************************************************** !*********************************************************************************** !* version 4.6 10/11/05 * !* - the SVD tolerance for singular values has been fine-tuned* !* to avoid problems whith unormalised data on polynomials * !* of degrees > 3 * !* - the filtering algorithm for non significant lines has * !* been changed with additional criteria and an iterative * !* procedure * !* - The Tuckey window has been deleted : it caused problems * !* at very low frequencies. The lines are then narrower and * !* the spectral resolution higher (cost : a 20% higher CPU) * !* - the independent variable (time) in input data no longer * !* needs to be sorted in increasing order * !* The reading procedure tests whether data is sorted either* !* in increasing or decreasing order or not sorted at all. * !* Arrays are sorted accordingly if needed * !* - portability improved with shorter format statement * !*********************************************************************************** !*********************************************************************************** !* version 4.5 12/07/05 * !* - major change in the routine arraymax used to determine * !* the strongest spectral line in a periodogram * !* it is replaced by routine powermax which locates the * !* five largest parabolic maximum. The maximum is found * !* after a parabolic interpolation of each of these lines * !* - for irregular sampling I have abandoned the application * !* of a spectral window. The routine has been optimised * !* - change in the frequency sampling to allow for line of * !* very irregular shape * !* - output of resid_00 before the first line is removed but * !* after trend and/or average removal * !* - correction of a minor mistake in the computation of the * !* power spectrum * !* - the portability has been improved by adhering more * !* to F90 standard (random_number, date_and_time) * !*********************************************************************************** !*********************************************************************************** !* version 4.4 05/04/05 * !* - major change in the routines waves to compute the * !* power spectrum : for irregular time sampling one takes * !* advantage of the regular sampling in frequency * !* to implement an evaluation of the sine and cosine * !* with a recurrence relation. * !* Program speeded up by a factor 5 ! * !* Running times w or w/o regular sampling are equivalent * !*********************************************************************************** !* version 4.3 28/03/05 * !* - new protections added in the search of parabolic maximum * !* - new routine get_top_bot to scan only parabolic * !* maximums and minimums. * !* - freqsearch routine fully restructured and rewritten * !* - program optimised when a periodic signal is searched. * !* - Routine periodsearch added to replace freqsearch when * !* a periodic signal is searched * !* - new flag in the setting :: flmulti * !*********************************************************************************** !* version 4.2 20/02/05 * !* - introduction of the module famouslib * !* - implicit none everywhere and all variable declared * !* - stricter adherance to f90 * !* - set of constants in the module header * !* - F90 version of Levenberg_Marquardt * !* - much improved testing of the significance of the lines * !* - detection and corrections for ghost lines * !* :: 2 lines closer than the spectral resolution * !* - bug fixed in the computation of the Jacobian * !* :: faster convergence and better stability * !* - many protections added against anomalous situations * !*********************************************************************************** !* version 4.1 10/09/04 * !* - Replacement of the all the linear LSQR with normal * !* equations by routines with singlular value decompostions * !* by routines with singlular value decompostions * !* - Improved tuning of the convergence conditions in the * !* iteratives processes * !* - optional output of the residuals and periodogram after * !* each frequency found for plotting purposes * !* - updated output layout * !*********************************************************************************** !* version 4.0 20/02/04 * !* - introduction of a model with mixed terms : * !* cos , t*cos, t^2*cos ... , sin , t*sin, t^2*sin * !*********************************************************************************** !*********************************************************************************** !* version 3.3 27/09/03 * !* -error in the size of array rms fixed. It could cause a * !* crash in the routine freqsol * !* -more reliable return at end of file in readsignal * !* -F90 syntax : continuation lines, comments, free format * !* -number of data points determined automatically * !* and storage via allocate statement * !* -new routine GETPARAM for the general parameters * !* -new structure for the program WAVES * !* -array size with the number of frequencies sampled * !* is computed internally and storage done with allocate * !* -upper achievable frequency determined internally from * !* the analysis of the time sampling (still in progress) * !* with the routine SAMPLING_HISTO * !* -computation of the SNR in the periodogram to test * !* the assumption of a quasi-periodic signal.This determines * !* the strength of the line in the periodogram. * !* Better than the rms when the signal has lines superimposed* !* to a non quasi-periodic part which is considered as noise * !* in the leastsquares analysis and increases the sigmas * !*********************************************************************************** !*********************************************************************************** !* version beta 3.2 delivered to J. Chapront 21/09/03 * !* Complete change of structure for the non linear search * !* A fast non linear LSR has been introduced and applied * !* at each new frequency to the whole set of frequencies * !* available. Levenberg-Marquart used in case of lack * !* of convergence. * !* Much better tests have been implemented to secure the * !* convergence * !* The programme is more robust, faster and much more * !* accurate at the end. * !* Lines are filtered out to eliminate non significant * !* lines. When lines are removed the two solutions * !* are provided. * !* Several subroutines have been added to facilitate the * !* development of the version with time dependant coefficients !*********************************************************************************** !*********************************************************************************** !* version beta 3.1 delivered to J. Chapront 17/09/03 * !* Introduction of lsqr_svd in freqsearch * !* SVD stands for : singular value decompostion * !*********************************************************************************** ! Third release September 2003 v3.0 ! ---------------------------------- ! ! Minor changes ! - improvement in the search of the line width in GETWIDTH ! - possibility to output the residual with IRESID = 1 in the settings ! ! Major changes ! - for simulated data possibility to read a time sampling in a file ! and then generate a simulated signal. READ_WINDOW ! - Optimisation when the time sampling is regular (arithmetic progression) ! The program determines whether the sampling is regular or not in SAMPLING ! Then specific routines are used. ! Tests have shown an improvement in runtime by a factor 5. ! To desactivate this option, search CALL SAMPLING and set flunif = .false. after the call. !*********************************************************************************** ! ! Second release February 2002 v2.0 ! ---------------------------- ! ! minor changes : ! - missing declaration of dimension in Freqsol fixed ! - protection of Arraymax against maximum found at the lower or upper indices ! major changes : ! - Important sign error in the computation of the jacobian fixed ! (this made the non linear fitting totally inefficient) ! - At each step, when a new frequency is found, a non linear optimization ! is performed on the initial signal to find the best amplitudes and ! frequencies. ! - A new version of the Levenberg-Marquart subroutine is used. Problems were ! found in the previous one. ! - A F90 compliant version of the linear leastsquares is used. ! ! ! First release January 2000 v1.0 ! --------------------------- ! Baseline version with periodograms and linear least-squares !************************************************************************************* ! !