Project

General

Profile

Support #22470

Use of Python constructs when possible to help context managers to properly manage resources.

Added by Lorena Lobato Pardavila 7 months ago. Updated 13 days ago.

Status:
Feedback
Priority:
Normal
Category:
-
Target version:
Start date:
08/23/2019
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Stakeholders:
Duration:

Description

We can improve the performance (and reduce code) with the with Statement and Context Managers when files are being managed.
Currently, close() is called on every file opened, and some times the file is in a function that may raise an exception or has multiple return paths, using for this try...except.
We could use 'with' statement that clarifies code that previously would use try...finally blocks to ensure all resources we use are properly cleaned up, regardless of if the code returns or an exception is thrown: context managers.
For example:

    # old file exists, check if same content
    fl = open(fname, 'r')
    try:
        old_data = fl.read()
    finally:
         fl.close()

We could do :

with open(fname,'r') as fl
 fd.read()

and not needed to close the files and the block is calling it automatically.

To DO: Review the code and do the replacement where this case is satisfied.

Note: It works with open(). But not with os.open() as it is just a wrapper for the lower-level POSIX syscall. It returns the file descriptor (a number) that represents the opened file and not a file object: so we cannot use it in this case.


Subtasks

Support #23166: Apply the TODOs from #22470 related to "with statement" compatibility New

History

#1 Updated by Lorena Lobato Pardavila 7 months ago

  • Target version set to v3_5

#2 Updated by Lorena Lobato Pardavila 6 months ago

  • Target version changed from v3_5 to v3_5_1

#3 Updated by Lorena Lobato Pardavila 6 months ago

  • Status changed from New to Work in progress
  • Tracker changed from Idea to Support

#4 Updated by Lorena Lobato Pardavila 5 months ago

Most of the changes were straightforward but there are two cases that we must have in consideration.

  1. Some parts of the changes are using file() [built-in constructor] instead of open().

Example:

 glidein_params_to_encrypt={}
            fd=file(tmpname, "a")
            nr_credentials=len(self.x509_proxies_data)
            if nr_credentials>0:
                glidein_params_to_encrypt['NumberOfCredentials']="%s"%nr_credentials

Basically, works the same way and open has been the canonical way to open a file (and hence create a file instance) for a long time, even on Python 2.x. As the documentation states it's preferable to use open() instead of invoking this constructor directly. So, when doing the changes for "with statement" , I propose to replace file() by open() at the same time.

  1. The other part to have in consideration is when catching exceptions, especially they are nested.

#5 Updated by Lorena Lobato Pardavila 4 months ago

  • Status changed from Work in progress to Feedback

it turned out that a lot of files have been changed requiring a deep review (some of the replacement are tricky and it might need discussion) from a caring soul 😊 .

I have proposed to review all the files in the next code review exposing the changes that I made and discuss them. Otherwise, if someone volunteers, I will assign the ticket to him for feedback.

#6 Updated by Lorena Lobato Pardavila 4 months ago

  • Target version changed from v3_5_1 to v3_6_1

After discussion, this will be reviewed in the next code review. It will be around mid- August

#7 Updated by Lorena Lobato Pardavila 3 months ago

  • Start date changed from 04/29/2019 to 08/23/2019
  • Due date set to 08/23/2019

due to changes in a related task: #23166

#8 Updated by Lorena Lobato Pardavila about 2 months ago

  • Target version changed from v3_6_1 to v3_6_2

#9 Updated by Lorena Lobato Pardavila about 1 month ago

  • Assignee deleted (Lorena Lobato Pardavila)

#10 Updated by Lorena Lobato Pardavila about 1 month ago

  • Target version changed from v3_6_2 to v3_6_1
  • Assignee set to Lorena Lobato Pardavila

#11 Updated by Lorena Lobato Pardavila 21 days ago

  • Target version changed from v3_6_1 to v3_6_2

#12 Updated by Lorena Lobato Pardavila 13 days ago

  • Assignee changed from Lorena Lobato Pardavila to Marco Mambelli


Also available in: Atom PDF