Explore topic-wise InterviewSolutions in .

A memory view OBJECT is returned by memoryview() a built-in function in standard library. This object is SIMILAR to a sequence but allows access to the internal data of an object that supports the buffer protocol without copying.

Just as in the C language, we access the memory using pointer variables; in PYTHON; we use memoryview to access its referencing memory.

Buffer Protocol allows exporting an interface to access internal memory (or buffer). The built-in TYPES bytes and bytearray supports this protocol and objects of these classes are allowed to expose internal buffer to allow to directly access the data. 

Memory view objects are required to access the memory directly instead of data duplication.

The memoryview() function has one object that supports the buffer protocol as argument.

This code creates a memoryview of bytesarray

A memoryview supports slicing and indexing to expose its data.

Slicing creates a subview

Python allows a function to be defined inside another function. Such functions are called nested functions. In such case, a certain variable is located in the order of local, outer, global and lastly built-in namespace order. The inner function can access variable declared in nonlocal scope.

Whereas the inner function is able to access the outer scope, inner function is able to use variables in the outer scope through closures. Closures maintain references to objects from the earlier scope. 

A closure is a nested function object that REMEMBERS values in ENCLOSING scopes even if they are not present in memory.

Following criteria must be met to create a closure in Python:

• We must have a nested function (function inside a function).
• The nested function must refer to a value defined in the enclosing function.
• The enclosing function must return the nested function.

Following code is an example of closure.

Duck typing is a special case of dynamic polymorphism and a characteristic found in all dynamically typed languages including Python. The premise of duck typing is a famous quote: "If it walks like a duck and it QUACKS like a duck, then it must be a duck".

In programming, the above duck test is used to determine if an object can be used for a particular purpose. With statically typed languages, suitability is determined by an object's type. In duck typing, an object's suitability is determined by the presence of certain METHODS and properties, RATHER than the type of the object itself. Here, the idea is that it doesn't actually matter what type data is - just whether or not it is possible to perform a certain operation.  

Take a simple case of ADDITION operation. Normally addition of similar objects is allowed, for example the addition of numbers or strings. In Python, addition of two integers, x+y actually does int.__add__(x+y) under the hood.

Hence, addition operation can be done on any objects whose class supports __add__() method. Python’s data model describes protocols of data types. For example sequence protocol stipulates that any class that supports iterator, len() and __getitem__() methods is a sequence. 

The simple example of duck typing given below demonstrates how any object may be used in any context, up until it is used in a way that it does not support:

Duck typing is possible in certain statically typed languages by providing extra type annotations  that instruct the compiler to arrange for type checking of classes to occur at run-time rather than COMPILE time, and include run-time type checking code in the compiled output.

Python supports data compression using various algorithms such as zlib, gzip, bzip2 and lzma. Python library also has modules that can manage ZIP and tar archives.

Data compression and decompression according to zlib algorithm is implemented by zlib module. The gzip module provides a simple interface to compress and decompress files just as very popular GNU utilities GZIP and GUNZIP.

Following example creates a gzip file by writing compressed data in it.

This will create “test.txt.gz” file in the current directory.

In order to read this compressed file:

Note that the gz file should be opened in wb and rb mode respectively for writing and reading.

The bzip2 compression and decompression is implemented by bz2 module. Primary interface to the module involves following three functions:

1. Open(): opens a bzip2 compressed file and returns a file object. The file can be opened as binary/text mode with read/write permissions. 
2. write(): the file should be opened in ‘w’ or ‘wb’ mode. In binary mode, it writes compressed binary data to the file. In normal text mode, the file object is wrapped in TetIOWrapper object to perform encoding.
3. read(): When opened in read mode, this function READS it and returns the uncompressed data.

Following code writes the compressed data to a bzip2 file

This will create test.bz2 file in the current directory. Any UNZIPPING tool will show a ‘test’ file in it. To read the uncompressed data from this test.bz2 file use the following code:

The Lempel–Ziv–Markov chain algorithm (LZMA) performs lossless data compression with a higher compression ratio than other algorithms. Python’s lzma module consists of classes and convenience functions for this purpose.

Following code is an example of lzma compression/decompression:

A ‘test.xz’ file will be created in the current working directory. To fetch uncompressed data from this file use the following code:

The ZIP is one of the most popular and old file formats used for archiving and compression. It is used by famous PKZIP application.

The zipfile module in Python’s standard library provides ZipFile() function that returns ZipFile object. Its write() and read() methods are used to create and read ARCHIVE.

To  read  data from a PARTICULAR file in the archive

Finally, Python’s tarfile module helps you to create a tarball of multiple files by applying different compression algorithms.

Following example opens a tar file for compression with gzip algorithm and adds a file in it.

Following code extracts the files from the tar archive, extracts all files and puts them in current folder. 

Counter class is defined in collections module, which is part of Python’s standard LIBRARY. It is defined as a subclass of built-in dict class. Main application of Counter object is to keep count of hashable objects. Counter object is a collection of KEY-value pairs, telling how many times a key has appeared in sequence or dictionary.

The Counter() function acts as a constructor. Without arguments, an empty Counter object is returned. When a sequence is given as argument, it results in a dictionary holding occurrences of each ITEM.

Counter object can be created directly by giving a dict object as argument which itself should be k-v pair of element and count.

The Counter object supports all the methods that a built-in dict object has. One important method is elements() which returns a chain object and returns an iterator of element and value repeating as many times as value.

Exceptions are generally fatal as the program TERMINATES instantly when an unhandled exception occurs. Warnings are non-fatal. A warning message is displayed but the program won’t terminate. Warnings generally appear if some deprecated usage of CERTAIN programming element like keyword/function/class etc. occurs.

Python’s builtins module defines Warnings class which is inherited from Exception itself. However, custom warning messages can be displayed with the help of WARN() function defined in built-in warnings module.

There are a number of built-in Warning subclasses. User defined subclass can also be defined.

Following code defines WarnExample class. When method1() is called, it issues a depreciation warning.

Output:

People often get confused between assertions and exceptions and often use the LATTER when the former should be used in a program (and vice versa).

The fundamental difference lies in the nature of the error to be caught and PROCESSED. Assertion is a statement of something that MUST be TRUE. If not, then the program is terminated and you cannot recover from it. Exception on the other hand is a runtime situation that can be recovered from.

Python’s assert statement checks for a condition. If the condition is true, it does nothing and your program just continues to execute. But if it evaluates to false, it raises an AssertionError exception with an optional error message.

The PROPER use of assertions is to inform developers about unrecoverable errors in a program. They’re not intended to signal expected error conditions, like “file not found”, where a user can take corrective action or just try again. The built-in exceptions like FileNotFoundError are appropriate for the same.

You can think of assert as equivalent to a raise-if statement as follows:

__debug__ constant is true by default. 

Python’s assert statement is a debugging aid, not a mechanism for handling run-time errors for which exceptions can be used.

Assertion can be globally disabled by setting __debug__ to false or starting Python with -O option.

Python 3 was introduced in 2008. It was meant to be backward incompatible although some features have later been backported to Python 2.7. Still because of vast differences in specifications, Python 2.x code won’t run on Python 3.x environment. 

However, Python 3 library comes with a utility to convert Python 2.x code. This module is called 2to3. Command Line usage of this TOOL is as below:

To test this usage, save following code as py2.py

Above code uses Python 2.x syntax. The print statement doesn’t require parentheses. Also raw_input() function is not available in Python 3.x. Hence when we convert above code, print() function will have parentheses and raw_input will CHANGE to input()

As we run above command, a diff against the original source file is printed as below:

2to3 can also write the needed modifications right back to the source file which is enabled with the -w flag. Changed version of script will be as shown.

The py2.py will be modified to be compatible with python 3.x syntax

The vars() function can be operated upon those objects which have __dict__ attribute. It returns the __dict__ attribute for a module, class, instance, or any other object having a __dict__ attribute. If the object doesn’t have the attribute, it raises a TypeError exception. 

The dir() function returns list of the attributes of any Python object. If no parameters are passed it returns a list of names in the current local scope.  For a class , it returns a list of its attributes as WELL as those of the base classes RECURSIVELY. For a module, a list of names of all the attributes, contained is returned.

When you access an object's attribute using the dot OPERATOR, python does a lot more than just LOOKING up the attribute in that objects dictionary.

Let us take a look at the following class:

The vars() of above class is similar to its __dict__  value

However, vars() of its object just shows instance attributes

The dir() function returns the attributes and methods of myclass as well as object class, the base class of all Python classes.

Whereas dir(a), the object of myclass returns instance variables of the object, methods of its class and object base class

Hashing ALGORITHM is a technique of OBTAINING an encrypted version from a string using a CERTAIN mathematical function. Hashing ensures that the message is securely transmitted if it is intended for a particular recipient only.

Several hashing ALGORITHMS are currently in use. Secure hash algorithms SHA1, SHA224, SHA256, SHA384, and SHA512 are defined by Federal Information Processing Standard (FIPS). 

RSA algorithm defines md5 algorithm.

The hashlib module in standard library provides a common interface to many different secure hash and message digest algorithms. 

hashlib.new(name,[data]) function in the module is a generic constructor that takes the string name of the desired algorithm as its first parameter.

The hexdigest() function returns a hashed version of string in hexadecimal symbols.

Instead of using new() as generic constructor, we can use a named constructor corresponding to hash algorithm. For example for md5 hash,

The hashlib module also defines algorithms_guaranteed and algorithms_available attributes.

The algorithms_guaranteed returns a set CONTAINING the names of the hash algorithms guaranteed to be supported by this module on all platforms. 

algorithms_available is a set containing the names of the hash algorithms that are available in the running Python interpreter.

A class having one or more abstract methods is an abstract class. A method on the other hand is called abstract if it has a dummy declaration and carries no DEFINITION. Such a class cannot be instantiated and can only be used as a base class to construct a subclass. HOWEVER, the subclass must implement the abstract methods to be a concrete class.

Abstract base classes (ABCs) introduce virtual subclasses. They don’t inherit from any class but are still recognized by isinstance() and issubclass() built-in functions.

For example, numbers module in standard library contains abstract base classes for all number DATA types. The INT, float or complex class will show numbers. Number class is its super class although it doesn’t appear in the __bases__ attribute.

The collections.abc module defines ABCs for Data structures like Iterator, Generator, Set, mapping etc.

The abc module has ABCMeta class which is a metaclass for defining custom abstract base class. In the following example Shape class is an abstract base class using ABCMeta. The shape class has area() method decorated by @abstractmethod decorator.

We now create a rectangle class derived from shape class and make it concrete by providing implementation of abstract method area()

If the abstract base class has more than one abstract method, the child class must implement all of them otherwise TypeError will be RAISED.

Both are built-in functions in STANDARD library. Both functions return sequence objects. However, bytes() returns an immutable sequence, whereas bytearray is a mutable sequence.

The bytes() function returns a bytes object that is an immutable sequence of integers between 0 to 255.

If string object is used with bytes() function, you should use encoding parameter, default is ‘utf-8’

Note that the bytes object has a literal representation too. Just prefix the string by b

On the other hand bytearray() returns a mutable sequence. If argument is an int, it INITIALIZES the array of given size with null.

However, if the argument is an iterable sequence, array size is equal to iterable count and items are initialized by elements of iterable. Following example shows bytearray object obtained from a list.

UNLIKE bytes object, a bytearray object doesn’t have literal representation.

Both bytes and bytearray objects support all normal sequence OPERATIONS like find and replace, join and count etc. 

Both objects also support LOGICAL functions such as isalpha(), isalphanum(), isupper(), islower() etc.

Case conversion functions are also allowed.

FieldStorage class is defined in cgi module of Python’s standard library. 

A http server can be configured to run executable scripts and programs stored in the cgi-bin directory on the server and render response in HTML form towards client. For EXAMPLE a Python script can be executed from cgi-bin directory.

However, the input to such a script may come from the client in the form of HTTP request either via its GET method or POST method.

In following HTML script, a form with two text input elements are defined. User data is sent as a HTTP request to a script mentioned as parameter of ‘action’ attribute with GET method.

In order to process the user input, Python’s CGI script makes use of FieldStorage object. This object is actually a dictionary object with HTML form elements as key and user data as value.

The fso object now contains form data which can be retrieved using usual dictionary notation.

FieldStorage class also defines the following methods:

• getvalue(): This method RETRIEVES value associated with form element. 
• getfirst(): This method returns only one value associated with form field name. The method returns only the first value in case more values were given under such name.
• getlist():This method always returns a list of values associated with form field name.

CGI, which stands for Common Gateway Interface, is a set of standards for the HTTP server to RENDER the output of executable scripts. CGI is one of the earliest technologies to be used for rendering dynamic content to the client of a web application. All major http web server software, such as Apache, IIS etc are CGI aware. With proper configuration, executable scripts written C/C++, PHP, Python, PERL etc can be executed on the server and the output is dynamically rendered in the HTML form to the client’s browser.

One important disadvantage of CGI is that the server starts a new PROCESS for every request that it RECEIVES from its clients. As a result, the server is likely to face severe bottlenecks and is slow.

WSGI (Web Server Gateway Interface) on the other hand prescribes a set of standards for communication between web servers and web applications written in Python based web application frameworks such as Django, Flask etc. WSGI specifications are defined in PEP 3333 (Python enhanced PROPOSAL)

WSGI enabled servers are designed to handle multiple requests concurrently. Hence a web application hosted on a WSGI server is faster.

There are many self-contained WSGI containers available such as Gunicorn and Gevent. You can also configure any Apache server for WSGI by installing and configuring mod_wsgi module. Many shared hosting services are also available for deploying Python web applications built with Django, Flask or other web frameworks. Examples are Heroku, Google App Engine, PythonAnywhere etc.

Whereas print() is a built-in FUNCTION that displays value of one or more arguments on the output console, pprint() function is defined in the pprint built-in module. The name stands for pretty print. The pprint() function PRODUCES an aesthetically good looking output of Python DATA structures. Any data structure that is properly parsed by the Python interpreter is ELEGANTLY formatted. It tries to keep the formatted EXPRESSION in one line as far as possible, but generates into multiple lines if the length exceeds the width parameter of formatting. 

In order to use it, first it must be imported from pprint module.

One of the unique features of pprint output is that it automatically sorts dictionaries before the display representation is formatted. 

The pprint module also has definition of PrettyPrinter class, and pprint() method belongs to it. 

According to Python’s standard documentation, compile() function returns code object from a string containing valid Python statements, a byte string or AST object.

A code object represents byte-compiled executable version of Python code or simply a BYTECODE. AST STANDS for Abstract syntax tree. The AST object is obtained by parsing a string containing Python code.

Syntax of compile() function looks as follows:

The source parameter can be a string, bytestring or AST object. Filename parameter is required if you are reading code from a file. The code object can be executed by exec() or eval() functions which is mode parameter. Other parameters are optional.

Thee FOLLOWING example compiles given string to a code object for it to be executed using exec() function.

The following example uses eval as mode parameter in compile() function. Resulting code object is executed using eval() function.

The AST object is first obtained by parse() function which is defined in ast module. It in turn is used to compile a code object.

The eval() function evaluates a single expression embedded in a string. The exec() function on the other hand RUNS the script embedded in a string. There may be more than one Python statement in the string.

Simple example of eval():

Simple example of exec()

eval() always RETURNS the RESULT of evaluation of expression or raises an error if the expression is incorrect. There is no return value of exec() function as it just runs the script.

Use of eval() may pose a security risk especially if user input is accepted through string expression, as  any harmful code may be input by the user.

Both eval() and exec() functions can perform operations on a code object returned by another built-in function compile(). Depending on mode parameter to this function, the code object is prepared for evaluation or execution.

Python’s built-in FUNCTION library provides input() and print() functions. They are default functions for console IO. The input() function receives data received from standard input stream i.e. keyboard and print() function sends data to standard output stream directed to standard output device i.e. DISPLAY monitor device.

In addition to these, we can make use of standard input and output stream objects defined in sys module. The sys.stdin OBJECT is a file like object capable of reading data from a console device. It in fact is an object of TextIOWrapper class capable of reading data. Just as built-in File object, read() and readline() methods are defined in TextIOWrapper.

Note that unlike input() function, the TRAILING newline character (\n) is not stripped here.

On the other hand sys.stdout object is also an object of TextIOWrapper class that is configured to write data in standard output stream and has write() and writelines() methods.

Python doesn’t implement data encapsulation principle of object oriented programming methodology in the sense that there are no access restrictions like private, public or protected members. Hence a traditional method (as used in Java) of accessing instance variables using getter and setter methods is of no AVAIL in Python.

Python recommends use of property object to provide easy interface to instance attributes of a class. The built-in property() function uses getter , setter and deleter methods to return a property attribute corresponding to a specific instance variable.

Let us first write a STUDENT class with name as instance attributes. We shall provide getname() and setname() methods in the class as below:

We can declare its object and access its name attribute using the above methods.

Rather than calling setter and getter function explicitly, the property object calls them whenever the instance variable underlying it is either retrieved or SET. The property() function uses the getter and setter methods to return the property object.

Let us add name as property object in the above User class by modifying it as follows:

The name property, returned by property() function hides private instance variable __name.  You can use property object directly. Whenever, its value is retrieved, it is internally called getname() method. On the other hand when a value is assigned to name property, internally setname() method is called.

The property() function also can have deleter method. A docstring can also be specified in its definition.

Python also has @property decorator that encapsulates property() function.

The term MONKEY patching is used to describe a technique used to dynamically MODIFY the behavior of a piece of code at run-time. This technique allows us to modify or extend the behavior of libraries, modules, classes or methods at runtime WITHOUT actually modifying the source code.

Monkey patching technique is useful in the following scenarios:

1. When we want to extend or modify the behavior of third-party or built-in libraries or methods at runtime without actually modifying the original code.

1. It is often found to be convenient when writing unit tests.

In PYTHON, classes are just like any other mutable objects. Hence we can modify them or their attributes including functions or methods at runtime.

In the following demonstration of monkey patching, let us first write a simple class with a method that adds two operands.

We can call add() method straightaway

However, we now need to modify add() method in the above class to accept a variable number of ARGUMENTS. What do we do? First define a newadd() function that can accept variable arguments and perform addition.

Now assign this function to the add() method of our MonkeyTest class. The modified behaviour of add() method is now available for all instances of your class.

Garbage COLLECTION is the mechanism of freeing up a computer’s memory by identifying and disposing of those objects that are no longer in use. C and C++ language compilers don’t have the capability of automatically disposing memory contents. That’s why disposal of memory resources has to be manually performed. Runtime environments of Java and Python have automatic garbage collection feature hence they provide efficient memory management.

Garbage collection is achieved by using different strategies. Java uses a tracing strategy WHEREIN it determines which objects should be ADDED to the garbage, by tracing the objects that are reachable by a chain of references from certain root objects. Those that are not reachable are considered as garbage and collecting them.

Python on the other hand uses a reference counting strategy. In this case, Python keeps COUNT of the number of references to an object. The count is incremented when a reference to it is created, and decremented when a reference is destroyed. Garbage is identified by having a reference count of zero. When the count reaches zero, the object's memory is reclaimed.

For example 0 is stored as int object and is assigned to variable a, thereby incrementing its reference count to 1. However, when a is assigned to another object, the reference count of 10 becomes 0 and is eligible for garbage collection.

Python’s garbage collector works periodically. To identify and collect garbage manually, we have gc module in Python’s library.  You can also set the collection threshold value. Here, the default threshold is 700. This means when the number of allocations vs. the number of deallocations is greater than 700, the automatic garbage collector will run.

The collect() FUNCTION identifies and collects all objects that are eligible.

The mechanism of execution of a Python program is similar to Java. A Python script is first COMPILED to a bytecode which in turn is executed by a platform specific Python virtual machine i.e. Python interpreter. However, unlike Java, bytecode version is not stored as a disk file. It is held temporarily in the RAM itself and DISCARDED after execution.

However, things are different when a script imports one or more MODULES. The imported modules are compiled to bytecode in the form of .pyc file and stored in a directory named as __pycache__.

Following Python script imports another script as module.

Run example.py from command line

You will find a __pycache__ folder created in current working directory with hello.cpython-36.pyc file created in it.

The advantage of this compiled file is that whenever hello module is imported, it need not be converted to bytecode again as LONG as it is not modified.

As mentioned above, when executed, .pyc file of the .py script is not created. If you want it to be explicitly created, you can use py_compile and compileall modules available in standard library.

To compile a specific script

This module can be directly used from command line

To compile all files in a directory

Following command line usage of compileall module is also allowed:

Where newdir is a folder under current working directory.

We normally come across LOGIN page of web application where a secret input such as a password or PIN is entered in a text field that masks the actual keys. How do we accept user input in a console based program? This is where the getpass module proves useful.

The getpass module provides a portable way to handle such password inputs securely so that the program can interact with the user via the terminal without showing what the user TYPES on the screen.

The module has the getpass() function which prompt the user for a password without echoing. The user is prompted using the string prompt, which defaults to 'Password: '.

Run above script as example.py from command line:

You can change the default prompt by explicitly defining it as argument to getpass() function

The getpass() function can also receive optional stream argument to redirect the prompt to a file LIKE object such as sys.stderr – the default value of stream is sys.stdout.

There is also a getuser() function in this module. It returns the “login name” of the user by CHECKING the environment variables LOGNAME, USER, LNAME and USERNAME.

ALTHOUGH Python is predominantly an object oriented programming language, it does have important functional programming capabilities INCLUDED in functools module.

One such function from the functools module is partial() function which returns a partial object. The object itself behaves like a function. The partial() function receives another function as argument and freezes some portion of a function’s arguments resulting in a new object with a simplified signature.

Let us consider the signature of built-in int() function which is as below:

This function converts a number or string to an integer, or returns 0 if no arguments are GIVEN.  If x is a number, returns x.__int__(). For floating point numbers, this truncates towards zero. If x is not a number or if base is given, then x must be a string.

We now use partial() function to create a callable that behaves like the int() function where the base argument defaults to 8.

In the following example, a user defined function power() is used as argument to a partial function square() by setting a default value on one of the arguments of the original function.

The functools module also defines partialmethod() function that returns a new partialmethod DESCRIPTOR which behaves like partial except that it is designed to be used as a method definition rather than being directly callable.

Instructions written in C/C++ are translated directly in a hardware specific machine code. On the other hand, Java/Python instructions are first converted in a hardware independent bytecode and then in turn the virtual machine which is specific to the hardware/operating system converts them in corresponding machine instructions.

The official distribution of Python hosted by https://www.python.org/ is called Cpython. The Python interpreter (virtual machine) software is written in C. We can call it as a C implementation of Python.

There are many alternatives to this official implementation. They are explained as below:

• Jython: Jython is a JRE implementation of Python. It is written in Java. It is designed to run on Java platform. Just as a regular Python module/package, a Jython program can import and use any Java class. Jython program also compiles to bytecode. One of the main advantages is that a user interface designed in Python can use GUI elements of AWT, swing or SWT package.
• Jython follows closely the standard Python implementation. Jython was created in 1997 by Jim Hugunin. Jython 2.0 was RELEASED in 1999. Current Jython 2.7.0 released in May 2015, corresponds to CPython 2.7. Development of Jython 3.x is under progress. It can be downloaded from https://www.jython.org/downloads.html
• IRONPYTHON: IronPython is a .NET implementation of Python. It is completely written in C#. Power of .NET framework can be easily harnessed by Python programs. Python’s rapid development tools are also easily embeddable in .NET applications. Similar to JVM, IronPython runs on Microsoft’s Common Language Runtime (CLR)
• IronPython’s current release, version 2.7.9 can be downloaded from https://ironpython.net/download/
• PyPy: PyPy is a fast, compliant alternative implementation of the Python language. It USES Just-in-Time compiler, because of which Python programs often run faster on PyPy. PyPy programs also consume less space than they do in CPython. PyPy is highly compatible with existing Python code.  PyPy comes by default with support for stackless mode, providing micro-threads for massive concurrency.
• Both CPython 2.x and 3.x compatible versions of PyPy are available for download on https://www.pypy.org/download.html

Python is an interpreter BASED language. Hence, a Python program is run by executing ONE statement at a time. Object version of the entire script is not created as in C or C++. As a result, Python interpreter must be available on the machine on which you intend to run Python program. On a Windows machine, a program is executed by following the command line:

The Python executable must be available in the system’s path.

Although the same command line can be used for running a program on Linux, there is one more option. On Linux, the Python script can be given execute permission by invoking chmod. However, for that to work, the first line in the script must specify the path to Python executable. This line, though it APPEARS as a comment, is called shebang.

The first line in above script is the shebang line and mentions the path to the Python executable. This script is now made executable by chmod command:

The Python script now becomes executable and doesn’t need executable’s name for running it.

However, the system still needs Python installed. So in that sense this is not self-executable. For this purpose, you can use one of the following utilities:

py2exe

cx-freeze

py2app

pyinstaller

The pyinstaller is very easy to use. First install pyinstaller with the help of pip

Then run the following command from within the directory in which your Python script is present:

In the same directory (here Desktop/example) a dist folder will be created. Inside dist, there is another directory with name of script (in this case example) in which the executable will be present. It can now be executed even if the target system doesn’t have Python installed.

This is how a self-executable Python application is bundled and DISTRIBUTED. The dist folder will CONTAIN all modules in the form of libraries.

Python’s built-in function library contains exec() function that is useful when it comes to dynamic execution of Python code. The prototype syntax of the function is as follows:

exec(object, GLOBALS, locals)

The object parameter can be a string or code object. The globals and locals PARAMETERS are optional. If only globals is provided, it must be a dictionary, which will be used for both the global and the local variables. If both are given, they are used for the global and local variables, respectively. The locals can be any mapping object.

In first example of usage of exec(), we have a string that contains a Python script.

Secondly, we have a Python script saved as example.py

The script FILE is read using built-in open() function as a string and then exec() function executes the code

As a third example, the string containing Python code is first compiled to code object using compile() function

Thus we can see how exec() function is useful for dynamic execution of Python code.

When you install Python and add Python executable to PATH environment variable of your OS, it is available for use from anywhere in the file system. Thereafter, while building a Python application, you may need Python libraries other than those shipped with the standard distribution (or any other distribution you may be using – such as Anaconda). Those libraries are generally installed using pip installer or conda package manager. These packages are installed in site-packages DIRECTORY under Python’s installation folder.

However, sometimes, requirement of a specific version of same package may sometimes be conflicting with other application’s requirements. Suppose you have application A that uses library 1.1 and application B requires 1.2 version of the same package which may have modified or deprecated certain functionality of its older version. Hence a system-wide installation of such package is likely to break application A.

To avoid this situation, Python provides creation of virtual environment. A virtual environment is a complete but isolated Python environment having its own library, site-packages and executables. This way you can work on DEPENDENCIES of a certain application so that it doesn’t clash with other or system wide environment.

For standard distribution of Python, virtual environment is set up using following command:

Here myenv is the name of directory/folder (along with its path. If none, it will be created under current directory) in which you want this virtual environment be set up. Under this directory, include, lib and scripts folders will be created. Local copy of Python executable along with other utilities such as pip are present in scripts folder. Script older ALSO has activate and DEACTIVATE batch files.

To activate newly created environment, run activate.bat in Scripts folder.

Name of the virtual environment appears before command prompt. Now you can invoke Python interpreter which create will local copy in the new environment and not system wide Python.

To deactivate the environment, simply run deactivate.bat

If using Anaconda distribution, you need to use conda install command.

This will create new environment under Anaconda installation folder’s envs directory.

Function is a reusable block of statements. WHENEVER called, it performs a certain PROCESS on parameters or arguments if passed to it. Following function receives two integers and returns their division

While calling the function, arguments must be passed in the same sequence in which they have been defined. Such arguments are called positional arguments. If it is desired to provide arguments in any other order, you must use keyword arguments as follows:

The formal names of arguments are used as keywords while calling the function. However, using keyword arguments is optional. Question is how to define a function that can be called only by using keywords arguments?

To force compulsory use of keyword arguments, use '*' as one parameter in parameter list. All parameters after * become keyword only parameters. Any parameters before * continue to be positional parameters.

In above example, both arguments are keyword arguments. Hence keyword arguments must be used even if you intend to pass values to arguments in the same order in which they have been defined.

A function can have MIX of positional and keyword arguments. In such case, positional arguments must be defined before ‘*’

In following example, the function has two positional and two keyword arguments. Positional arguments may be passed using keywords also. However, positional arguments must be passed before keyword arguments.

The dataclasses module is one of the most recent modules to be added in Python's standard library. It has been INTRODUCED since version 3.7 and defines @dataclass decorator that automatically generates following methods in a user defined class:

The dataclass function decorator has following prototype:

All the arguments are of Boolean value. Each decides whether a corresponding magic method will be automatically generated or not.

The 'init' parameter is True by default which will automatically generate __init__() method for the class.

Let us define Employee class using dataclass decorator as follows:

The auto-generated __init__() method is like:

Auto-generation will not happen if the class contsins explicit definition of __init__() method.

The repr argument is ALSO true by default.  A __repr__() method will be generated. Again the repr string will not be auto-generated if class provides explicit definition.

The eq argument forces __eq__() method to be generated. This method gets called in response to equals comparison operator (==). Similarly other operator overloading magic methods will be generated if the order argument is true (the default is False), __lt__(), __le__(), __gt__(), and __ge__() methods will be generated, they implement comparison operators < <= > ans >= respectively.

If unsafe_hash argument is set to False (the default), a __hash__() method is generated according to how eq and frozen are set.

frozen argument: If true (the default is False), it emulates read-only frozen instances.

Other useful functions in this module are as follows:

asdict():

This function converts class instance into a dictionary object.

astuple():

This function converts class instance into a tuple object.

make_dataclass():

This function creates a new dataclass from the list of tuples given as fields argument.

SQL injection is the technique of inserting malicious SQL statements in the user input thereby gaining unauthorized access to database and causing harm in an interactive data driven application.

Most computer applications including Python applications that involve database interaction use relational database products that work on SQL queries. Python has standardized database adapter specifications by DB-API standard. Python standard distribution has sqlite3 module which is a DB-API compliant SQLite driver. For other databases the module must be installed. For example, if using MySQL, you will have to install PyMySQL module.

Standard procedure of EXECUTING SQL queries is to first establish connection with database SERVER, obtain cursor object and then execute SQL query string. Following describes the sequence of steps for SQLite database

For example, following code displays all records in products table whose category is ‘Electronics’

However, if price is taken as user input then the query string will have to be constructed by inserting user input variable.

if input is Electronics, the query will be built as select * from products where ctg=’Electronics’But a malicious input such as following is given enter category:electronics;drop table products;Now this will cause the query to be as follows:

• select * from products where ctg=’Electronics’;drop table products;
• Subsequent cur.execute(qry) STATEMNT will inadvertently delete products table as well!
• To prevent this DB-API recommends use of parameterized queries using ? Placeholder.
• cur.execute(‘select * from products where ctg=?’, (catg))
• In some database adapter modules such as mysql, other placeholder may be used
• cur.execute(‘select * from products where ctg=%s’, (catg))

In such prepared statements, database engine checks validity of interpolated data and MAKES sure that undesirable query is not executed, preventing SQL injection attack.

This module implements pseudo-random number GENERATOR (PRNG) technique that uses the Mersenne Twister algorithm. It produces 53-bit precision floats and has a period of 2**19937-1. The module functions depend on the basic function random(), which generates a random float uniformly in the range 0.0 to 1.0.  

The Mersenne Twister is one of the most extensively TESTED random number generators in existence. However, being completely deterministic, it is not suitable for cryptographic purposes.

Starting from Python 3.6, the secrets module has been added to Python standard library for generating cryptographically strong random numbers (CSPRNG) suitable for managing data such as passwords, account authentication, security tokens, etc.

It is recommended that, secrets should be used in preference to the default pseudo-random number generator in the random module, which is designed for modelling and simulation, not security or cryptography.

Here’s how a secure password can be generated using secrets module. Let the password have one character each from group of uppercase characters, lowercase characters, digits and special characters.

Characters in these groups can be obtained by using following attributes defined in string module.

We now randomly CHOOSE one character from each group using choice() function.

Finally these four characters are shuffled randomly and joined to produce a cryptographically secure password.

Python has a built-in file object that represents a disk file. The file API has write() and read() methods to store data in Python objects in computer files and read it back.

To save data in a computer file, file object is first obtained from built-in open() function.

Here ‘w’ as mode parameter of open() function indicates that file is opened for WRITING data. To read back the data from file open it with ‘r’ mode.

However, the file object can store or retrieve only string data. For other type of objects, they must be either converted to string or their byte representations and store in binary files opened with ‘wb’ mode.

This type of manual conversion of objects in string or byte format (and vice versa) is very cumbersome. Answer to this situation is object serialization.

Object serialization means transforming it in a format that can be stored, so as to be able to deserialize it later, recreating the original object from the serialized format. Serialization may be done disk file, byte string or can be TRANSMITTED via network sockets. When serialized data is BROUGHT back in a form identical to original, the mechanism is called de-serialization.

Pythonic term for serialization is pickling while de-serialization is often referred to as unpickling. Python’ library contains pickle module that provides dumps() and loads() functions to serialize and deserialize Python objects.

Following code shows a dictionary object PICKLED using dumps() function in pickle module.

Original state of object is retrieved from pickled representation using loads() function.

The serialized representation of data can be persistently stored in disk file and can be retried back using dump() and load() functions.

Pickle data format is Python-specific in nature. To serialize and deserialize data to/from other standard formats Python library provides json, csv and xml modules.

As we can see, serialized data can be persistently stored in files. However, file created using write() method of file object does store data persistently but is not in serialized form. Hence we can say that serialized data can be stored persistently, but converse is not necessarily true.

In programming language context, a first-class object is an entity that can perform all the operations generally available to other scalar data types such as integer, float or string.

As they say, everything in Python is an object and this applies to function (or method) as well.

As you can SEE, a built-in function as well as user defined function is an object of function class.

Just as you can have built-in data type object as argument to a function, you can also declare a function with one of its arguments being a function itself. FOLLOWING example illustrates it.

Return value of a function can itself be ANOTHER function. In following example, two functions add() and multiply() are defined. Third function addormultiply() returns one of the first two depending on arguments passed to it.

Thus a function in Python is an object of function class and can be passed to other function as argument or a function can return other function. Hence Python function is a high order object.

Array is a collection of more than one element of similar data type in C/C++/Java etc. Python doesn't have any built-in equivalent of array. Its List/tuple data types are COLLECTIONS, but items in it may be of different types.

Python's array module emulates C type array. The module defines 'array' class whose constructor requires two arguments:

The typecode determines the type of array. Initializer should be a sequence with all elements of matching type.

The array module defines typecodes attribute which RETURNS a STRING. Each character in the string represents a type code.

Following statement creates an integer array object:

The initializer may be a byte like object. Following example builds an array from byte representation of string.

Some of the useful methods of the array class are as follows:

extend():

This method appends items from list/tuple to the end of the array. Data type of array and iterable list/tuple must MATCH; if not, TypeError will be raised.

fromfile():

This method reads data from the file object and appends to an array.

tofile():

This method write all items to the file object which must have write mode enabled.

append():

This method appends a new item to the end of the array

fromlist():

This method appends items from the list to array. This is equivalent to for x in list: a.append(x)

insert():

Insert a new item in the array before specified position

pop():

This method returns item at given index after removing it from the array.

remove():

This method removes first occurrence of given item from the array.

Any class in Python code, whether built-in or user defined, is an object of ‘type’ class which is called a 'metaclass'.  A metaclass is a class whose objects are also classes. This relationship can be understood by the following diagram:

Python library's built-in function type() returns class to which an object belongs.

Same thing applies to object of a user defined class

Interestingly, class is also an object in Python. Each class, both built-in class and a user defined class is an object of type class

Can be rephrased and mentioned at the START ALONG with the diagram and the tables above can be used as an example instead.

The type class is the default metaclass. The type() function used above takes an object as ARGUMENT. The type() function's three argument variation is used to define a class dynamically. Its prototype is as follows:

The function returns class that has been CREATED dynamically. Three arguments of type function are:

Following statement creates user dynamically using type() function:

We can now use this class in the same manner as we do when it is defined using class statement:

Similarly you can add instance METHOD dynamically.

Moreover, this class can be used to create an inherited class as well.

As mentioned above, type is the default metaclass. A class that it inherited from type is a custom metaclass.

Using this custom metaclass to create new class as follows:

Newly created class can even be used as base for other.

The custom metaclass is a type of both the classes. 

Construction of subclass of user defined metaclass can be customized by overriding __new__ magic method.

In following example, we declare  a metaclass (NewMetaClass) having __new__() method. It returns instance of new class which in turn is initialized by __init__() method as usual in the derived class.

When we declare an object of newclass and print its attributes, following output is displayed.

Output:

We can thus customize the metaclass and creation of its subclasses.

Obviously, print() is most commonly used Python statements and is used to display the value of one or more objects. However, it accepts other arguments other than objects to be displayed.

The function can have any number of objects as positional arguments, all separated by commas. Other arguments are optional but must be used as KEYWORD arguments.

By default, the non-keyword arguments that is the list of objects is displayed with WHITESPACE as a separator. However, you can define separator symbol with the help of sep argument.

Second keyword argument controls the end of line character. By default output of each print() statement terminates with newline character or ‘\n’. If you wish you can change it to any desired character.

The file argument decides the output stream to which result of print() is directed. By default it is sys.stdout which is standard output 

• device – the primary display device of the system. It can be changed to any file LIKE object such as a DISK file, bytearray, network stream 
• etc – any object possessing write() method.

The buffer argument is false by default, but if the flush keyword argument is true, the stream is forcibly flushed.

Errors detected during execution are called EXCEPTIONS. In Python, all exceptions must be INSTANCES of a class that derives from BaseException. Python library defines many subclasses of BaseException. They are called built-in exceptions. Examples are TypeError, ValueError etc.

When USER code in try: block raises an exception, program control is thrown towards except block of corresponding predefined exception type. However, if the exception is different from built-in ones, user can define a customized exception class. It is called user defined exception.

Following example defines MyException as a user defined exception.

Built-in exceptions are raised implicitly by the Python interpreter. However, exceptions of user defined type must be raised explicitly by using raise keyword.

The try: block accepts a number from user and raises MyException if the number is beyond the range 0-100. The except block then processes the exception object as defined.

Output:

<P>There are three distinct SITUATIONS in Python script where ‘as’ keyword is appropriately used.

1. to define alias for imported package, module or function.

1. to define a context manager variable:

1. as argument of exception:

When an exception occurs inside the try block, it may have an ASSOCIATED value known as the argument defined after exception type with ‘as’ keyword. The type of the argument depends on the exception type. The except clause may specify a variable after the exception name. The variable is bound to an exception instance with the arguments stored in instance.args.

Python script containing a definition of function, class or module usually contains a ‘docstring’ which appears as a comment but is treated as a description of the corresponding object.

The doctest module uses the docstring. It searches docstring text for interactive Python sessions and verifies their output if it is exactly as the output of the function itself.

The doctests are useful to check if docstrings are up-to-date by verifying that all interactive examples still work as documented. They also PERFORM regression testing by verifying interactive examples from a test file or a test object. Lastly they serve as an important tool to write tutorial documentation for a package.

Following code is a simple example of doctest. The script defines add() function and embeds various test cases in docstring in interactive manner.  The doctest module defines testmod() function which runs the interactive examples and checks output of function for mentioned test cases with the one mentioned in docstring. If the output matches, test is passed otherwise it is failed. 

Save the above script as example.py and run it from command line. To GET verbose output of doctests use -v option.

Test examples can be put in a separate textfile and subjected as argument to testfile() function instead of using testmod() function as above.

Unit testing is one of the various software testing methods where individual units/ components of software are tested. Unit testing refers to tests that verify the functionality of a specific section of code, usually at the function level. 

The UNITTEST module is a part of Python’s standard library. This unit testing framework was originally inspired by JUNIT. Individual units of source code, such as functions, methods, and class are tested to determine whether they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. Unit tests are short code FRAGMENTS created by programmers during the development process.

TEST case is the smallest unit of testing. This checks for a specific response to a particular set of inputs. unittest provides a base class, TestCase, which may be used to create new test cases.

test suite is a collection of test cases, test suites, or both and is used to aggregate tests that should be executed together. Test suites are implemented by the TestSuite class.

Following code is a simple example of unit test using TestCase class.

First let us define a function to be tested. In the following example, add() function is to be tested.

Next step is to Create a testcase by subclassing unittest.TestCase.

Inside the class, define a test as a method with name starting with 'test'.

Each test call assert function of TestCase class. The assertEquals() function compares result of add() function with arg2 ARGUMENT and throws assertionError if comparison fails.

Run above script. Output shows that the test is passed.

Possible outcomes are:

• OK: test passes
• FAIL: test doesn’t pass and AssertionError exception is raised.
• ERROR: test raises exception other than AssertionError

The unittest module also has a command line interface to be run using Python’s -m option.

The PIP is a very popular package management utility used to install and manage Python packages especially those hosted on Python’s official package repository called Python Package index(PyPI).

The pip utility is distributed by default in standard Python’s distributions with versions 2.7.9 or later or 3.4 or later. For other Python versions, you may have to obtain it by first downloading get-pip.py from https://bootstrap.pypa.io/get-pip.py and running it.

First, check the installed VERSION of pip on your system.

Note that for Python 3.x, the pip utility is named as pip3.

Most common usage of pip is to install a certain package using the following syntax:

In order to install a specific version of the package, version number is followed by name

Use --UPGRADE option to upgrade the local version of the package to the latest version on PyPi.

Many a time, a PROJECT needs many packages and they may be having a lot of dependencies. In order to install all of them in one go, first prepare a list of all the packages needed in a file called ‘requirements.txt’ and ask pip to use it for batch installation.

Uninstalling a package is easy

If the package needs to be installed from a repository other than PyPI, use –index-url option.

There is ‘search’ option to search for a specific package available.

The show option displays additional information of a package

The freeze option shows installed packages and their versions.

IP address is a string of decimal (or hexadecimal) numbers that forms a unique identity of each device connected to a computer network that uses INTERNET Protocol for data communication. The IP address has two purposes: identifying host and location addressing.

To start with IP addressing scheme uses 32 bits comprising of four octets each of 8 bits having a value equivalent to a decimal number between 0-255. The octets are separated by dot (.). For example 111.91.41.196

This addressing scheme is called Ipv4 scheme.  As a result of rapid increase in the number of devices directly connected to the internet, this scheme is being GRADUALLY replaced by Ipv6 version.

An IPv6 address uses hexadecimal digits to represent a string of unique 128 bit number. Each position in an IPv6 address represents four bits with a value from 0 to f. The 128 bits are divided into 8 groupings of 16 bits each separated by colons.

Example: 2001:db8:abcd:100::1/64

The IPv6 address always uses CIDR notation to determine how many of the leading bits are used for network identification and rest for host/interface identification.

Python's ipaddress module provides the capability to create, manipulate and operate on IPv4 and IPv6 addresses and networks. Following factory FUNCTIONS help to CONVENIENTLY create IP addresses, networks and interfaces:

ip_address():

This function returns an IPv4Address or IPv6Address object depending on the IP address passed as argument. 

ip_network():

This function returns an IPv4Network or IPv6Network object depending on the IP address passed as argument.

ip_interface():

Polymorphism is an important feature of object oriented programming. It comes into play when there are commonly named methods across subclasses of an abstract base class. 

Polymorphism is the ability to present the same interface for differing underlying forms. This allows functions to use objects of any of these polymorphic classes without needing to be AWARE of distinctions across the classes.

If class B inherits from class A, it doesn’t have to inherit everything about class A, it can do some of the things that class A does differently. It is most commonly used while dealing with inheritance. Method in Python class is implicitly polymorphic. Unlike C++ it need not be MADE explicitly polymorphic.

In the following example, we first define an abstract class CALLED shape. It has an abstract method area() which has no implementation.

Two subclasses of shape - Rectangle and Circle are then CREATED. They provide their respective implementation of inherited area() method.

Finally we set up objects with each of these classes and put in a collection. 

Run a for loop over a collection. A generic call to area() method calculates the area of respective shape.

Python’s import mechanism makes it possible to use CODE from one script in another. Any Python script (having .py extension) is a module. Python offers more than one way to import a module or class/function from a module in other script.

Most COMMON practice is to use ‘import’ keyword. For example, to import functions in math module and CALL a function from it:

Another way is to import specific function(s) from a module instead of populating the namespace with all contents of the imported module. For that purpose we need to use ‘from module import function’ syntax as below:

Use of wild card ‘*’ is also permitted to import all functions although it is discouraged; instead, BASIC import statement is preferred.

Python library also has __import__() built-in function. To import a module using this function:

Incidentally, import keyword internally calls __import__() function.

Lastly we can use importlib module for importing a module. It contains __import__() function as an alternate implementation of built-in function. The import_module() function for dynamic IMPORTS is as below:

A Python script has .py extension and can have DEFINITION of class, function, constants, variables and even some executable code. Also, any Python script can be imported in another script using import keyword.

However, if we import a module having certain executable code, it will automatically get EXECUTED even if you do not intend to do so. That can be avoided by identifying __name__ property of module.

When Python is running in interactive mode or as a script, it sets the value of this special VARIABLE to '__main__'. It is the name of the scope in which top level is being executed.

From within a script also, we find value of __name__ attribute is set to '__main__'. Execute the following script.

However, for imported module this attribute is set to the name of the Python script. (excluding the extension .py)

If we try to import example module in test.py script as follows:

However we wish to prevent the print statement in the executable part of example.py. To do that, identify the __name__ property of the module. If it is __main__ then only the print statement will be executed if we modify example.py as below:

Now the output of test.py will not show the name of the imported module.

The acronym JSON stands for JavaScript Object Notation based on a subset of the JavaScript language. It is a lightweight data interchange format which is easy for humans to use and easy for machines to parse.

Python library provides json MODULE for converting Python objects to JSON format and VICE versa. It contains dumps() and loads() FUNCTIONS to serialize and de-serialize Python objects.

The json module has dump() and load() functions that perform serialization and deserialization of Python object to a File like object such as disk file or network stream. In the following example, we store JSON string to a file using load() and retrieve Python dictionary object from it. Note that File object must have relevant ‘write’ and ‘read’ permission.

The ‘testjson.txt’ file will SHOW following contents:

To load the json string from file to Python dictionary we use load() function with file opened in ‘R’ mode.

The json module also provides object oriented API for the purpose with the help of JSONEncode and JSONDecoder classes.

The module also defines the relation between Python object and JSON data type for interconversion, as in the following table:

In Python, range is one of the built-in objects. It is an immutable sequence of integers between two numbers separated by a fixed interval. The range object is obtained from built-in range() function.

The range() function has two signatures as follows:

The function can have one, two or three integer arguments named as start, stop and step respectively. It returns an object that produces a sequence of integers from start (inclusive) to stop (EXCLUSIVE) by step.

In case only one ARGUMENT is given it is treated as stop value and start is 0 by default, step is 1 by default.

If it has two arguments, first is start and second is stop. Here step is 1 by default.

If the function has three arguments, they will be used as start, stop and step parameters. The range will contain integers from start to step-1 separated by step.

Elements in the range object are not automatically displayed. You can cast range object to list or tuple.

The most common use of range is to TRAVERSE and process periodic sequence of integers using ‘for’ loop. The following statement produces the SQUARE of all numbers between 1 to 10.

The range object has all methods available to a sequence, such as len(), index() and count()

It is also possible to obtain an iterator out of the range object and traverse using next() function.

While the two names sound similar, they are in fact entirely different.

Python is a general purpose programming language that is on top of the popularity charts among programmers worldwide. IPython, on the other hand, is an interactive command-line terminal for Python. 

Standard Python runs in the interactive MODE by invoking it from the command terminal to start a REPL (Read, Evaluate, Print and Loop) with Python prompt >>>

IPython is an enhanced Python REPL with much more features as compared to standard Python shell. Some of the features are given below:

• SYNTAX highlighting
• Stores the history of interactions
• Offers features such as Tab completion of keywords, variables and function names
• Has object introspection feature
• Has magic command system for controlling Python environment
• Can be embedded in other Python programs
• Acts as a main kernel for Jupyter notebook.

IPython has been DEVELOPED by FERNANDO Perez in 2001. Its current version is IPython 7.0.1, that runs on Python 3.4 version or higher. IPython has spun off into Project Jupyter that provides a web based interface for the IPython shell.

IPython is bundled with Anaconda distribution. If you intend to install IPython with standard Python, use PIP utility

Or better still, you can install Jupyter. IPython, being one of its dependencies, will be installed automatically.

The following diagram shows the IPython console.

The Jupyter notebook frontend of IPython looks as below:

At first, it gives the impression that given expression is invalid because of railing comma. But it is perfectly valid and Python interpreter doesn’t raise any error.

As would be expected, expression without railing comma declares an int object.

Note that use of parentheses in the REPRESENTATION of tuple is optional. Just comma separated values FORM a tuple.

Hence a number (any object for that matter) followed by comma, and without parentheses forms a tuple with single element.

As mentioned above omitting comma is a REGULAR assignment of int variable.