Topic: Deleting a fiscal year problem

Hello,
i am using frontaccounting since 2014 and i did not delete any records. Now i have some problems. Because i have too many records in db. For example gl_trans table has 5.5millions of records. Now i decided to delete fiscal years but it cannot possible to delete from frontaccounting user interface becouse of timeout limitations. I wrote cli script with using delete_this_fiscalyear method in mainteince_db.inc. I can close a fiscal year and then i can delete this fiscal year.
After deletion, customer blances has been changed. I investigated what is the problem. I found something but i don't know how to solve this problem exactly.
If i delete 2014 records and customer has invoice in this year but it has paid in 2015, total customer balance is changing. As if the customer paid more than its debt but the payment belongs to invoice in 2014. I hope i could explain clearly.
Now i'm using 2.3.26 version and I can not upgrade to 2.4.x becacuse i have lot of records. Upgrade process takes very very long time like infinite.
I have to delete previous fiscal years but i cant.
I am waiting your reply.
Thanks in advance.

Re: Deleting a fiscal year problem

I cannot speak to the correctness of your cli script approach, but you may want to instead consider modifying the applicable FA timeouts.   You can extend the maximum php timeout as in this post.   The ajax timeout is less straightforward because there is no option to set the timeout.  You could try changing js/inserts.js:

//  '.ajaxsubmit,.editbutton,.navibutton': // much slower on IE7
    'button.ajaxsubmit,input.ajaxsubmit,input.editbutton,button.editbutton,button.navibutton':
    function(e) {
            e.onclick = function() {
                if (validate(e)) {
                    save_focus(e);
                    var asp = e.getAttribute('aspect')
                    if (asp && (asp.indexOf('process') !== -1))
                        JsHttpRequest.request(this, null, 600000); // ten minutes for backup
                    else
                        JsHttpRequest.request(this);
                }
                return false;
            }
    },

to

//  '.ajaxsubmit,.editbutton,.navibutton': // much slower on IE7
    'button.ajaxsubmit,input.ajaxsubmit,input.editbutton,button.editbutton,button.navibutton':
    function(e) {
            e.onclick = function() {
                if (validate(e)) {
                    save_focus(e);
                    var asp = e.getAttribute('aspect')
                    if (asp && (asp.indexOf('process') !== -1))
                        JsHttpRequest.request(this, null, 600000); // ten minutes for backup
                    else
                        JsHttpRequest.request(this. null, 600000); // ten minutes for all buttons such as delete?
                 }
                return false;
            }
    },

but I am just guessing that the fiscal year delete button executes that code.

Re: Deleting a fiscal year problem

I am giving the way of my solution. Because I tried one functionality before for the users who are coming to FA from other software's.  Like they wanted to keep the journal of previous customer opening balances as journal entry. And when the customer make payment record the payment against that journal entries.

So the solution is to separate the invoices which are paid on 2015 and invoiced in 2014.  And than we have to summarise the balance of it and make one journal entry.  And change the allocation of invoices to that newly created journal entry to take effect. Let's say if you have 50 invoices in 2014 and paid in 2015. All these invoices needs to be removed and created one journal entry with all its balance.  After that we have to reallocate the payments against that journal entry.  Like this. We can remove all the other entries like stock balance in stock moves,

You have to remove related entries in
audit trail
Bank trans
Comments,
Cust allocations
Debtor trans
Debtor trans details
Gl trans
Journal
Grn_items, batch
Purch_orders
Purch_order_details
Sales orders
Sales orders details
Stock moves
Supp trans
Supp trans details
Trans tax details
Wo orders,

And other tables if anything affecting the yearly data. 
May be If you see tother

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

@Braath Waate
Thank you for timeout solution. CLI script exactly same as handle_delete method in fiscalyears.php. I'am using delete_this_fiscalyear method in fiscalyears_db.inc (I wrote wrong in first message)

@kvvaradha
Thank you for your answer. It's hard to determine this kind of invoices. Below you can see total invoices i have year by year. I have to write another script for this therefore study db structure and relation of records more.

2014 - 21511
2015 - 40067
2016 - 64865
2017 - 98377
2018 - 114123
2019 - 152206

Re: Deleting a fiscal year problem

The existing delete fiscal year function already handles to delete in other entries. And you don't need to study much of it. But you have to study something to create journal entry for opening balance in sales and purchase.  Before deleting it on customer invoice and supplier invoice.  We have to make this function to create opening balance. And than you can allow it to delete the entries. I can understand it would take more time to delete it. Probably . You can wrote a cron job to run it for every 10mins and in a certain length of invoices and data. The system won't freeze or fail to remove. You can run it in free time like your night time or in holidays. So you can achieve it in less time and you don't need to worry about timeouts. There you have more data. So you have to write a separate function and call it inside FA. Or call it inside cron job. That will do the process.

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

I'll try your advice. If i can create journal entry for opening balance to every customer (or supplier) who has balance at end of year, i think problem is solved. For without timeout limitation, cli script is best in my opinion. i will need it once a year anyway.

Re: Deleting a fiscal year problem

One more thing to do. The allocation of old entries will be deleted so i have to make new allocation for opening balance and unallocated payments.

Re: Deleting a fiscal year problem

That's good idea. But you have to know which invoices are allocated. With that it's good for you to reallocate it.  Hope you got good solution. Once you implement it successfully or issues. Just drop your feedbacks and hints

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

Yes you're right. Thanks for the idea smile
I'll inform you when finished.

Re: Deleting a fiscal year problem

@joe: Need to refurbish the end of year routines to make sure that the client balances are automatically made along with unpaid and partially paid invoice data for allocation of payments in the subsequent year. This needs to be done for suppliers as well. The appropriate stock moves too need to be taken care of.

Re: Deleting a fiscal year problem

This is a difficult task. I remember writing these routines years back and we really tried to get it all at that time.
I will try to look into it asap.

Joe

Re: Deleting a fiscal year problem

Hello again.

I have now went through the deleting fiscal year procedure. I have found no problems here.
Here is a list over what happens when processing deleting of a fiscal year:

Delete Fiscalyear

1. Backup is done
2. All sales orders and details, up to last date, that are fully delivered are removed. Attachments are removed.
3. All purchase orders and details, up to last date, that are fully received are removed. Attachments are removed.
4. All GRN receivals, up to last date, are removed.
5. All debtor trans and details, up to last date, that are fully allocated are removed. Attachments are removed.
6. All cust allocations, up to last date, are removed.
7. All supp trans and details, up to last date, that are fully allocated are removed. Attachments are removed.
8. All supp allocations, up to last date, are removed.
9. All workorders, up to last date, and closed.
   a. All workorder issues are removed. Attachments are removed.
   b. All workorder manufacture are removed. 
   c. All workorder requirements are removed.
   d. Workorders removed. Attachments are removed.
10. All stock moves, up to last date, are removed and converted to a closing stock move.
11. All voided items, up to last date, are removed.
12. All trans tax details, up to last date, are removed.
13. All exchange rater, up to last date, are removed.
14. All cust and supp sub gl transactions are retrieved by account, removed and an end transaction are inserted. 
15. All bank trans, up to last date, are removed and converted to a closing balance trans.
16. All ausit trail, up to last date, are removed.
17. All comments, up to last date, are removed.
18. Finally the fiscalyear is removed.

Joe

Re: Deleting a fiscal year problem

How ever there is a problem with payment of next year for previous year invoice.

Let's take his example. He issued several invoices  at the end of last year and collected the payment in next year.  Which means the system needs to make one journal entry for a customer at least to record the invoice balances as opening balance and that should be allocated in the next year.

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

Oh yes, I see. The alloc field got stamped in the next year and when the invoice was not allocated by end of last year.
I will have to look at the allocation routines instead. Thanks.

Joe

Re: Deleting a fiscal year problem

The allocations has been revised and should now work. Committed to stable repo. The fixed file can be downloaded here.

Joe

Re: Deleting a fiscal year problem

@joe: Thanks for solution. I have been working on it for a few days. Your solution is ver simple and i 'll try asap. Will it work for payed before and invoiced next year transactions?

Re: Deleting a fiscal year problem

@joe, your solution is good. Instead of rewriting it as journal entry. We keep the invoices alive and it's allocations alive. So it will be removed if they remove next fiscal year. 

@madosk, This is definitely a time saving way for you to implement it.  Hope your problem fixed with this. And post an reply after using this updated program. It would be a solved solution for future referring users.

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

Updated Wiki.

Re: Deleting a fiscal year problem

I have compatibility problem. I 'm using 2.3.26 but commited file compatible with 2.4.x.
2.3.26 neither has add_journal method nor 0_journal table.
I removed this part of code and tried to delete year but unfortunately customer balance calculated wrong.
I'm investigating why balance is wrong. I'll inform you more if i find something.

20 (edited by madosk 05/20/2020 02:49:50 pm)

Re: Deleting a fiscal year problem

After deletion year 2014 customer has records in debtor_trans table. For example;

debtor_trans
trans_no: 15496
type: 12
tran_date: 2014-09-29
ov_amount: 785
alloc: 785

cust_allocations
amt: 335
date_alloc: 2014-09-29
trans_no_from: 15496
trans_type_from: 12
trans_no_to: 14931
trans_type_to: 10   

amt: 450
date_alloc: 2014-09-29
trans_no_from: 15496
trans_type_from: 12
trans_no_to: 15472
trans_type_to: 10

But 15472, 14931 transactions do not exist in debtor_trans. Why this payment didn't deleted? whereas it payed in 2014 and allocated to invoices in 2014.

By the way invoices are still alive which are processed in 2014 and allocated in 2015. This part is working correct i think but payments part is not.

21 (edited by kvvaradha 05/20/2020 03:34:34 pm)

Re: Deleting a fiscal year problem

Looks like this is a strange issue. I feel the below code is missing the cust allocation.

  while ($row = db_fetch($result))
    {
        if ($row['type'] == ST_SALESINVOICE)
        {
            $deliveries = get_sales_parent_numbers($row['type'], $row['trans_no']);
            foreach ($deliveries as $delivery)
            {
                $sql = "DELETE FROM ".TB_PREF."debtor_trans_details WHERE debtor_trans_no = $delivery AND debtor_trans_type = ".ST_CUSTDELIVERY;
                db_query($sql, "Could not delete debtor trans details");
                $sql = "DELETE FROM ".TB_PREF."debtor_trans WHERE trans_no = $delivery AND type = ".ST_CUSTDELIVERY;
                db_query($sql, "Could not delete debtor trans");
                delete_attachments_and_comments(ST_CUSTDELIVERY, $delivery);
            }        
        }    
        $sql = "DELETE FROM ".TB_PREF."cust_allocations WHERE trans_no_from = {$row['trans_no']} AND trans_type_from = {$row['type']}";
        db_query($sql, "Could not delete cust allocations");
.
.
.
}

And may be it should be like this

while ($row = db_fetch($result))
    {
        if ($row['type'] == ST_SALESINVOICE)
        {
            $deliveries = get_sales_parent_numbers($row['type'], $row['trans_no']);
            foreach ($deliveries as $delivery)
            {
                $sql = "DELETE FROM ".TB_PREF."debtor_trans_details WHERE debtor_trans_no = $delivery AND debtor_trans_type = ".ST_CUSTDELIVERY;
                db_query($sql, "Could not delete debtor trans details");
                $sql = "DELETE FROM ".TB_PREF."debtor_trans WHERE trans_no = $delivery AND type = ".ST_CUSTDELIVERY;
                db_query($sql, "Could not delete debtor trans");
                delete_attachments_and_comments(ST_CUSTDELIVERY, $delivery);
            }        
                      $sql = "DELETE FROM ".TB_PREF."cust_allocations WHERE trans_no_to = {$row['trans_no']} AND trans_type_to = {$row['type']}";  // newly added 
        db_query($sql, "Could not delete cust allocations");  // newly added. 
        }    
        $sql = "DELETE FROM ".TB_PREF."cust_allocations WHERE trans_no_from = {$row['trans_no']} AND trans_type_from = {$row['type']}";
        db_query($sql, "Could not delete cust allocations");
}

I have added two more lines added newly to collect the respective invoice allocation entires delete. Like the same for supplier allocations.

If it helps feedback us to fix it in core.

Subscription service based on FA
HRM CRM POS batch Themes

Re: Deleting a fiscal year problem

This fix only delete allocation. Payment must be deleted too.

$sql = "SELECT trans_no, type FROM ".TB_PREF."debtor_trans WHERE tran_date <= '$to' AND 
        (ov_amount + ov_gst + ov_freight + ov_freight_tax + ov_discount) = 
        (SELECT SUM(amt) FROM ".TB_PREF."cust_allocations WHERE date_alloc <= '$to' AND trans_no = trans_no_to AND
        type = trans_type_to)";

This sql can not find the payment to delete.

Re: Deleting a fiscal year problem

Hi all again,
Unfortunately i couldn't solve this problem yet. I modified sql to find invoices, payments and credit notes which allocated in year to delete.

$sql = "SELECT trans_no, type 
            FROM ".TB_PREF."debtor_trans 
            WHERE tran_date <= '$to' 
            AND (round((ov_amount + ov_gst + ov_freight + ov_freight_tax + ov_discount), 3) = round((SELECT SUM(amt) FROM ".TB_PREF."cust_allocations WHERE date_alloc <= '$to' AND trans_no = trans_no_to AND    type = trans_type_to), 3) 
            OR round((ov_amount + ov_gst + ov_freight + ov_freight_tax + ov_discount), 3) = round((SELECT SUM(amt) FROM ".TB_PREF."cust_allocations WHERE date_alloc <= '$to' AND trans_no = trans_no_from AND    type = trans_type_from), 3))";

But it's not enough to find right transactions to delete. Still I have to separate some invoices and payment.
So I have an invoice (1800$) invoiced in 2015 and i have a payment (800$) paid in 2015. This payment is fully allocated to this invoice but remaining balance of this invoice (1000$) paid in 2016.
This sql find 800$ payment to delete because this payment is fully allocated in 2015 but invoice is still alive because it's not fully allocated in 2015.  I have to keep this payment to find right balance at the end of 2015.
How can i do this? Hope i could explain the problem which i couldn't solve.

Thanks in advance.