1 (edited by itronics 12/28/2007 09:36:19 am)

Topic: Two bugs in backup module

Hi joe

I found 2 bugs in admin/db/maintenance_db.inc:

1. Backup files made on MySQL (as of version 5.0.32) are unusable. This is because of bad interpretation of NULL fields in response of 'SHOW COLUMNS xxx' clause at db_export() function. These fields are filled with with 'NO' value (not empty field) at least in above MySQL version.

2. At start of function db_import() there are 2 lines with str_replace() calls which seems to be garbage from any earlier FA version. Anyway with these two lines uncommented database is corrupted after export/import cycle.

BTW I think available range of backup (all tables vs data for one company) could depend on current user privileges. However this implies separate per user folders.

PS. If you want I can send diff for above changes.

janusz

Re: Two bugs in backup module

Hello,
Thanks for your contribution. Regarding #1. Do you have a solution to that?
Regarding #2. These 2 lines with str_replace() is neccessary. The first line is for empty chart of accounts. They are marked with a 0_ table prefix and are replaces with the current company table prefix. The second line is for the Report Generator SQL file. A table prefix of Y_ should be replaced with the 0_ prefix. Could this be rewritten to work with your test?

Kind regards
Joe

Re: Two bugs in backup module

Hi joe
Ad 1. Diff folows.

Ad 2. Nope. Report generator stores all reports (for all users and all companies) in one MySQL table xx_reports. This is not good solution, because all users has access to all other company data via 'Test the SQL statement' button. This is serious security issue.

On the other way if backup system is available for Admins only this implies backing up whole database, not only for one company. In one database can coexist table sets with 0_, 1_ etc prefixes and with no prefix at all. So why the code changes only tables with prefix 0_, but the others are backed up with table names leaved intact?

So is the diff:

--- /var/www/frontaccounting/admin/db/maintenance_db.inc    2007-10-01 10:13:28.000000000 +0200
+++ /var/www/fa/admin/db/maintenance_db.inc    2007-12-27 21:36:05.000000000 +0100
@@ -125,8 +125,8 @@
    {
        $line = trim($line);

-        $line = str_replace("0_", $connection["tbpref"], $line);
-        $line = str_replace("Y_", "0_", $line);
+//        $line = str_replace("0_", $connection["tbpref"], $line);
+//        $line = str_replace("Y_", "0_", $line);
        // the last line did not belong to a 'create' sql query
        if (!$table)
        {
@@ -397,7 +397,7 @@
                    for ($k = 0; $k < $nf = db_num_fields($res2); $k++)
                    {
                        // identify null values and save them as null instead of ''
-                        if ($field_type[$k] != "" && $row2[$k] == "")
+                        if ($field_type[$k] != "" && $field_type[$k] != "NO" && $row2[$k] == "")
                            $out .= "NULL";
                        else
                            $out .= "'" . db_escape($row2[$k]) . "'";

Re: Two bugs in backup module

Thank you Janusz for your help.
#1 is ok. Will be fixed asap.
#2. A temporarily fix to the Report Generator is to limit the access to Admins only. Page_security 15. I think the second str_replace line was mistakenly implemented during development of the Report Generator. It is not neccessary. But the first line with str_replace with replacement of 0_ to currentry table-pref is neccessary due to importing new chart of accounts when creating new companies. These charts have a prefix of 0_. It shouldn't create any troubles at least as I see it. If it does, we have to find another way of adding the table-prefixes into the tables. The problem here is that all the various chart of accounts have been implemented with this 0_ prefix. What do you say to this? When we have solved this we can update the CVS repository.
Again, did you see my answer to your PDF presentation? If we are sure about that we can include this also.

Kind regards
Joe

Re: Two bugs in backup module

Hello  joe
May be my previous post was not clear, but sorry - english is not my native language. I'll try again.

joe wrote:

#2. A temporarily fix to the Report Generator is to limit the access to Admins only.

Yes, it is. This suggests backup of whole database in a manner which makes possible restoring all table sets for all companies from admin generated sql file. But:
1. db_import() function doesn't restore 0_ prefix.
2. what about such case:  you have in database 2 sets of tables for 2 companies with prefixes: 0_, 1_ . Assuming you are logged on admin account with '1_' prefix, after export you have sql backup file with  statements creating table sets with prefixes: '1_' and '1_' (once again). How will look your database after restoring from such backup file? Answer: as before restore operation, or worse - dataset '1_' will be updated with '0_'.


joe wrote:

The problem here is that all the various chart of accounts have been implemented with this 0_ prefix. What do you say to this? When we have solved this we can update the CVS repository.

This is no problem. If you want to use backup module to make 'reference' sql files on development system - you can. Play with database with only one table set with prefix '0_', logging from its account. Then none prefix substitution is necessary.

joe wrote:

Again, did you see my answer to your PDF presentation? If we are sure about that we can include this also.

I've answered in that thread.

Regards
janusz

Re: Two bugs in backup module

Hello again janusz,
I'm not sure if I understand your answer.
When you choose the backup/restore in the setup tab, only the current company is backed up. If this company has a table prefix of 0_, the backup file will show up as: dbname_0_Ymd_Hi.sql, where dbname is the name of the database, _0 is the table prefix. It is important that you select the correct sql file with the correct table prefix when you later restore this file on the company.
If there is no table prefix the backup file will show: dbname_Ymd_Hi.sql. In this case you would chose the correct dbname when restoring.
String replacing a 0_ with 0_ will result in 0_
String replacing a 0_ with an empty string will result in an empty string (no table prefix).
The only cases that the str_replace line will be executed are when you replace a 0_ with 0_. (backup/restore), and when creating a new company. It will either replace the 0_ with the selected table prefix or an empty string if no prefixes.

So I really see no problem here if you take these precautions when restoring the databases.

/Joe

Re: Two bugs in backup module

Hi joe
I double checked the problem and found that you are partially right.
If the backup tool is used on account with any standard prefix or to install fresh new company from reference files (sql statements with prefix 0_)  all is correct.

But the problem exists. If you are logged on account with no prefix db_export() makes backup file for the whole database (all table sets). And here problems described in my earlier posts arise.

Below is new patch which remove this issue. One issue still exists, but must be changed in report generator module. database xx_reports is backed up only for user which has empty prefix.

Regards
janusz



--- /var/www/frontaccounting/admin/db/maintenance_db.inc    2007-10-01 10:13:28.000000000 +0200
+++ /var/www/fa/admin/db/maintenance_db.inc    2007-12-30 13:20:45.000000000 +0100
@@ -126,7 +126,6 @@
        $line = trim($line);

        $line = str_replace("0_", $connection["tbpref"], $line);
-        $line = str_replace("Y_", "0_", $line);
        // the last line did not belong to a 'create' sql query
        if (!$table)
        {
@@ -321,7 +320,8 @@
     $all_tables = array();
     while($row = db_fetch($res))
     {
-        if ($conn["tbpref"] == "" || strpos($row['Name'], $conn["tbpref"]) !== false)
+        if (($conn["tbpref"] == '' && !preg_match('/[0-9]+_/', $row['Name']))
+         || ($conn["tbpref"] != '' && strpos($row['Name'], $conn["tbpref"]) !== false))
             $all_tables[] = $row;
     }

Re: Two bugs in backup module

Fixed in the CVS repository at Sourceforge. Thanks janusz.

/Joe

Re: Two bugs in backup module

I am sure I have the case related to described in this tread - I made a backup of 0_ company (version 1.12 build 31) and now can not restore it. The database is getting corrupt as result of the restore.

I have setup v. 2.0 (beta build 22) but the issue remains - can't restore the backup.

Is there anything I can do about it? I have damn lots of transactions made, so will hate the idea of manual re-work.

Please help!

regarsd,
Alexey

Re: Two bugs in backup module

Hello Alexey,
The version 1.12 and v2.0 are not compatible. There are changes in the database structure. Therefore you can not restore the backup from v1.12 on the newly installed v2.0.
You can, however, run the update_db.php script on the companies if you have installed in the same folder as the 1.12. This should be done before logging into the companies.

/Joe

Re: Two bugs in backup module

hi Joe,

Thank you for the response - i had problems restoring the company still having version 1.12. Somebody told me that script file is inconsistent (data is conflicting with table descriptions - specifically, having NULL values where "NOT NULL" is declaired).

Is there anything I can do with my backup file? Shall I install later version compatible with 1.12? Or shall I set up the 1.12 and try to import (again) the data there?

Thank you in advance!!

Re: Two bugs in backup module

Please update to version 1.16. I think there was a bug related to this, reported by itronics and he fixed it in 1.16.
There are no changes in the database structure between 1.12 and 1.16. Just follow the update.html instructions.

/Joe